You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Dmitriy Ryaboy <dv...@gmail.com> on 2012/06/01 11:02:58 UTC

Re: this nested foreach worked in 0.8.1, no longer works in 0.10

Do you have relations ned sent, opened, etc, elsewhere in the script?

On May 31, 2012, at 12:09 PM, Steve Bernstein <St...@deem.com> wrote:

> We upgraded from Pig 0.8.1 to 0.10 and the following nested foreach no longer works:
> 
> actionBagGrouped = GROUP actionBag BY (deal_id,month);
> dealCounts = FOREACH actionBagGrouped {
>                sent = FILTER actionBag BY action=='sent';
>                opened = FILTER actionBag BY action=='opened';
>                clicked = FILTER actionBag BY action=='clicked';
>                purchased = FILTER actionBag BY action=='purchased';
> 
>                GENERATE
>                                FLATTEN(group),
>                                COUNT(sent) AS sentCount,
>                                COUNT(opened) AS openedCount,
>                                COUNT(clicked) AS clickedCount,
>                                COUNT(purchased) AS purchasedCount;
> };
> I get:
> "Invalid scalar projection: action : A column needs to be projected from a relation for it to be used as a scalar"
> Banging my head against a wall-the problem is with the nest op sent = Filter...
> 
> Help!
> Thanks-
> Steve

Re: this nested foreach worked in 0.8.1, no longer works in 0.10

Posted by Jagat <ja...@gmail.com>.
Hello Dmitriy,

My apology for missing braces , actually i removed extra stuff from the
code to make it general.

as i am using just to give alias name.

You can read it as SUBSTRING inside group By clause for 0.10

On Wed, Jun 6, 2012 at 8:03 AM, Dmitriy Ryaboy <dv...@gmail.com> wrote:

> Steve,
> Scalar projection errors usually happen when you use "foo.bar" syntax
> instead of "foo::bar" (and a relation called foo happens to exist) or,
> possibly, if there is a relation you create inside a nested foreach,
> and another one with identical name outside. The former doesn't seem
> to be present in the snippet you provided. The latter has been fixed
> iirc (and I think it actually manifested in a different error, don't
> remember off the top of my head). Would really need to see the
> complete script to reproduce the error and see what's going on.
>
> Jagat -- are you sure the code you pasted is what you are getting
> errors on? For starters, it doesn't have balanced parenthesis. It's
> also unclear to me what you are trying to do with the "as" clause.
>
> D
>
> On Tue, Jun 5, 2012 at 5:53 AM, Subir S <su...@gmail.com> wrote:
> > It was very nice to know answers to these, since it will help while
> > upgrading from 0.8.1 to 0.10.0 pig.
> >
> > On Sun, Jun 3, 2012 at 11:38 PM, Jagat <ja...@gmail.com> wrote:
> >
> >> Hello everyone ,
> >>
> >> I also came across one bug in 0.10.0 would like to confirm following
> >> behaviour which i observed.
> >>
> >> myGroup = group myData by ( SUBSTRING(NAME,0,1) as FIRST_LETTER;
> >>
> >> Assume NAME is present in myData
> >>
> >> Syntax error, unexpected symbol at or near 'SUBSTRING'
> >>
> >> The above code used to work in 0.8 version , with 0.10 version it gives
> >> error above.
> >>
>

Re: this nested foreach worked in 0.8.1, no longer works in 0.10

Posted by Dmitriy Ryaboy <dv...@gmail.com>.
Steve,
Scalar projection errors usually happen when you use "foo.bar" syntax
instead of "foo::bar" (and a relation called foo happens to exist) or,
possibly, if there is a relation you create inside a nested foreach,
and another one with identical name outside. The former doesn't seem
to be present in the snippet you provided. The latter has been fixed
iirc (and I think it actually manifested in a different error, don't
remember off the top of my head). Would really need to see the
complete script to reproduce the error and see what's going on.

Jagat -- are you sure the code you pasted is what you are getting
errors on? For starters, it doesn't have balanced parenthesis. It's
also unclear to me what you are trying to do with the "as" clause.

D

On Tue, Jun 5, 2012 at 5:53 AM, Subir S <su...@gmail.com> wrote:
> It was very nice to know answers to these, since it will help while
> upgrading from 0.8.1 to 0.10.0 pig.
>
> On Sun, Jun 3, 2012 at 11:38 PM, Jagat <ja...@gmail.com> wrote:
>
>> Hello everyone ,
>>
>> I also came across one bug in 0.10.0 would like to confirm following
>> behaviour which i observed.
>>
>> myGroup = group myData by ( SUBSTRING(NAME,0,1) as FIRST_LETTER;
>>
>> Assume NAME is present in myData
>>
>> Syntax error, unexpected symbol at or near 'SUBSTRING'
>>
>> The above code used to work in 0.8 version , with 0.10 version it gives
>> error above.
>>

Re: this nested foreach worked in 0.8.1, no longer works in 0.10

Posted by Subir S <su...@gmail.com>.
It was very nice to know answers to these, since it will help while
upgrading from 0.8.1 to 0.10.0 pig.

On Sun, Jun 3, 2012 at 11:38 PM, Jagat <ja...@gmail.com> wrote:

> Hello everyone ,
>
> I also came across one bug in 0.10.0 would like to confirm following
> behaviour which i observed.
>
> myGroup = group myData by ( SUBSTRING(NAME,0,1) as FIRST_LETTER;
>
> Assume NAME is present in myData
>
> Syntax error, unexpected symbol at or near 'SUBSTRING'
>
> The above code used to work in 0.8 version , with 0.10 version it gives
> error above.
>

Re: this nested foreach worked in 0.8.1, no longer works in 0.10

Posted by Jagat <ja...@gmail.com>.
Hello everyone ,

I also came across one bug in 0.10.0 would like to confirm following
behaviour which i observed.

myGroup = group myData by ( SUBSTRING(NAME,0,1) as FIRST_LETTER;

Assume NAME is present in myData

Syntax error, unexpected symbol at or near 'SUBSTRING'

The above code used to work in 0.8 version , with 0.10 version it gives
error above.

Re: this nested foreach worked in 0.8.1, no longer works in 0.10

Posted by Dmitriy Ryaboy <dv...@gmail.com>.
Yeah, anything that reproduces the error would be great. Thanks!

D

On Thu, Jun 7, 2012 at 10:35 AM, Steve Bernstein
<St...@deem.com> wrote:
> Thanks Dmitriy.  The full script is probably more than you want (and more than I think I'd want to send), but let me create a "subset" of it, run it to check that it's otherwise sound and that the error replicates, then send that.  Would that be okay?
>
> -----Original Message-----
> From: Dmitriy Ryaboy [mailto:dvryaboy@gmail.com]
> Sent: Wednesday, June 06, 2012 11:56 AM
> To: user@pig.apache.org
> Subject: Re: this nested foreach worked in 0.8.1, no longer works in 0.10
>
> Any chance you can send the full script?
>
> On Jun 6, 2012, at 11:44 AM, Steve Bernstein <St...@deem.com> wrote:
>
>> Meanwhile, I'm moving forward running this script on 0.8.1 with no issues.  Probably going to back off on using 0.10 for now and wait for a more stable release.
>>
>> -----Original Message-----
>> From: Steve Bernstein [mailto:Steve.Bernstein@deem.com]
>> Sent: Sunday, June 03, 2012 9:25 AM
>> To: user@pig.apache.org
>> Subject: RE: this nested foreach worked in 0.8.1, no longer works in
>> 0.10
>>
>> Since this seems to do with project, before I wrote to the list, I'd already tried actionBag.sent and actionBagGrouped::actionBag.sent.  No joy.  Is this a bug?  Seems like others would have stumbled across it already.
>> SB
>> ________________________________________
>> From: Subir S [subir.sasikumar@gmail.com]
>> Sent: Saturday, June 02, 2012 4:04 AM
>> To: user@pig.apache.org
>> Subject: Re: this nested foreach worked in 0.8.1, no longer works in
>> 0.10
>>
>> Can you try  sent = FILTER actionBag BY  actionBag.action=='sent';
>>
>> Although not sure why it does not work. I also use pig 0.8.1 and similar constructs work for me.
>>
>> On Fri, Jun 1, 2012 at 10:55 PM, Steve Bernstein
>> <St...@deem.com>wrote:
>>
>>> No, all those relations embedded in the nested foreach are unique to
>>> it, not used elsewhere.
>>> SB
>>>
>>> -----Original Message-----
>>> From: Dmitriy Ryaboy [mailto:dvryaboy@gmail.com]
>>> Sent: Friday, June 01, 2012 2:03 AM
>>> To: user@pig.apache.org
>>> Cc: user@pig.apache.org
>>> Subject: Re: this nested foreach worked in 0.8.1, no longer works in
>>> 0.10
>>>
>>> Do you have relations ned sent, opened, etc, elsewhere in the script?
>>>
>>> On May 31, 2012, at 12:09 PM, Steve Bernstein
>>> <St...@deem.com>
>>> wrote:
>>>
>>>> We upgraded from Pig 0.8.1 to 0.10 and the following nested foreach
>>>> no
>>> longer works:
>>>>
>>>> actionBagGrouped = GROUP actionBag BY (deal_id,month); dealCounts =
>>>> FOREACH actionBagGrouped {
>>>>               sent = FILTER actionBag BY action=='sent';
>>>>               opened = FILTER actionBag BY action=='opened';
>>>>               clicked = FILTER actionBag BY action=='clicked';
>>>>               purchased = FILTER actionBag BY action=='purchased';
>>>>
>>>>               GENERATE
>>>>                               FLATTEN(group),
>>>>                               COUNT(sent) AS sentCount,
>>>>                               COUNT(opened) AS openedCount,
>>>>                               COUNT(clicked) AS clickedCount,
>>>>                               COUNT(purchased) AS purchasedCount; };
>>>> I get:
>>>> "Invalid scalar projection: action : A column needs to be projected
>>>> from
>>> a relation for it to be used as a scalar"
>>>> Banging my head against a wall-the problem is with the nest op sent
>>>> =
>>> Filter...
>>>>
>>>> Help!
>>>> Thanks-
>>>> Steve
>>>

RE: this nested foreach worked in 0.8.1, no longer works in 0.10

Posted by Steve Bernstein <St...@deem.com>.
Thanks Dmitriy.  The full script is probably more than you want (and more than I think I'd want to send), but let me create a "subset" of it, run it to check that it's otherwise sound and that the error replicates, then send that.  Would that be okay?  

-----Original Message-----
From: Dmitriy Ryaboy [mailto:dvryaboy@gmail.com] 
Sent: Wednesday, June 06, 2012 11:56 AM
To: user@pig.apache.org
Subject: Re: this nested foreach worked in 0.8.1, no longer works in 0.10

Any chance you can send the full script?

On Jun 6, 2012, at 11:44 AM, Steve Bernstein <St...@deem.com> wrote:

> Meanwhile, I'm moving forward running this script on 0.8.1 with no issues.  Probably going to back off on using 0.10 for now and wait for a more stable release.
> 
> -----Original Message-----
> From: Steve Bernstein [mailto:Steve.Bernstein@deem.com]
> Sent: Sunday, June 03, 2012 9:25 AM
> To: user@pig.apache.org
> Subject: RE: this nested foreach worked in 0.8.1, no longer works in 
> 0.10
> 
> Since this seems to do with project, before I wrote to the list, I'd already tried actionBag.sent and actionBagGrouped::actionBag.sent.  No joy.  Is this a bug?  Seems like others would have stumbled across it already.
> SB
> ________________________________________
> From: Subir S [subir.sasikumar@gmail.com]
> Sent: Saturday, June 02, 2012 4:04 AM
> To: user@pig.apache.org
> Subject: Re: this nested foreach worked in 0.8.1, no longer works in 
> 0.10
> 
> Can you try  sent = FILTER actionBag BY  actionBag.action=='sent';
> 
> Although not sure why it does not work. I also use pig 0.8.1 and similar constructs work for me.
> 
> On Fri, Jun 1, 2012 at 10:55 PM, Steve Bernstein
> <St...@deem.com>wrote:
> 
>> No, all those relations embedded in the nested foreach are unique to 
>> it, not used elsewhere.
>> SB
>> 
>> -----Original Message-----
>> From: Dmitriy Ryaboy [mailto:dvryaboy@gmail.com]
>> Sent: Friday, June 01, 2012 2:03 AM
>> To: user@pig.apache.org
>> Cc: user@pig.apache.org
>> Subject: Re: this nested foreach worked in 0.8.1, no longer works in
>> 0.10
>> 
>> Do you have relations ned sent, opened, etc, elsewhere in the script?
>> 
>> On May 31, 2012, at 12:09 PM, Steve Bernstein 
>> <St...@deem.com>
>> wrote:
>> 
>>> We upgraded from Pig 0.8.1 to 0.10 and the following nested foreach 
>>> no
>> longer works:
>>> 
>>> actionBagGrouped = GROUP actionBag BY (deal_id,month); dealCounts = 
>>> FOREACH actionBagGrouped {
>>>               sent = FILTER actionBag BY action=='sent';
>>>               opened = FILTER actionBag BY action=='opened';
>>>               clicked = FILTER actionBag BY action=='clicked';
>>>               purchased = FILTER actionBag BY action=='purchased';
>>> 
>>>               GENERATE
>>>                               FLATTEN(group),
>>>                               COUNT(sent) AS sentCount,
>>>                               COUNT(opened) AS openedCount,
>>>                               COUNT(clicked) AS clickedCount,
>>>                               COUNT(purchased) AS purchasedCount; }; 
>>> I get:
>>> "Invalid scalar projection: action : A column needs to be projected 
>>> from
>> a relation for it to be used as a scalar"
>>> Banging my head against a wall-the problem is with the nest op sent 
>>> =
>> Filter...
>>> 
>>> Help!
>>> Thanks-
>>> Steve
>> 

Re: this nested foreach worked in 0.8.1, no longer works in 0.10

Posted by Dmitriy Ryaboy <dv...@gmail.com>.
Any chance you can send the full script?

On Jun 6, 2012, at 11:44 AM, Steve Bernstein <St...@deem.com> wrote:

> Meanwhile, I'm moving forward running this script on 0.8.1 with no issues.  Probably going to back off on using 0.10 for now and wait for a more stable release.
> 
> -----Original Message-----
> From: Steve Bernstein [mailto:Steve.Bernstein@deem.com] 
> Sent: Sunday, June 03, 2012 9:25 AM
> To: user@pig.apache.org
> Subject: RE: this nested foreach worked in 0.8.1, no longer works in 0.10
> 
> Since this seems to do with project, before I wrote to the list, I'd already tried actionBag.sent and actionBagGrouped::actionBag.sent.  No joy.  Is this a bug?  Seems like others would have stumbled across it already.
> SB
> ________________________________________
> From: Subir S [subir.sasikumar@gmail.com]
> Sent: Saturday, June 02, 2012 4:04 AM
> To: user@pig.apache.org
> Subject: Re: this nested foreach worked in 0.8.1, no longer works in 0.10
> 
> Can you try  sent = FILTER actionBag BY  actionBag.action=='sent';
> 
> Although not sure why it does not work. I also use pig 0.8.1 and similar constructs work for me.
> 
> On Fri, Jun 1, 2012 at 10:55 PM, Steve Bernstein
> <St...@deem.com>wrote:
> 
>> No, all those relations embedded in the nested foreach are unique to 
>> it, not used elsewhere.
>> SB
>> 
>> -----Original Message-----
>> From: Dmitriy Ryaboy [mailto:dvryaboy@gmail.com]
>> Sent: Friday, June 01, 2012 2:03 AM
>> To: user@pig.apache.org
>> Cc: user@pig.apache.org
>> Subject: Re: this nested foreach worked in 0.8.1, no longer works in 
>> 0.10
>> 
>> Do you have relations ned sent, opened, etc, elsewhere in the script?
>> 
>> On May 31, 2012, at 12:09 PM, Steve Bernstein 
>> <St...@deem.com>
>> wrote:
>> 
>>> We upgraded from Pig 0.8.1 to 0.10 and the following nested foreach 
>>> no
>> longer works:
>>> 
>>> actionBagGrouped = GROUP actionBag BY (deal_id,month); dealCounts = 
>>> FOREACH actionBagGrouped {
>>>               sent = FILTER actionBag BY action=='sent';
>>>               opened = FILTER actionBag BY action=='opened';
>>>               clicked = FILTER actionBag BY action=='clicked';
>>>               purchased = FILTER actionBag BY action=='purchased';
>>> 
>>>               GENERATE
>>>                               FLATTEN(group),
>>>                               COUNT(sent) AS sentCount,
>>>                               COUNT(opened) AS openedCount,
>>>                               COUNT(clicked) AS clickedCount,
>>>                               COUNT(purchased) AS purchasedCount; 
>>> }; I get:
>>> "Invalid scalar projection: action : A column needs to be projected 
>>> from
>> a relation for it to be used as a scalar"
>>> Banging my head against a wall-the problem is with the nest op sent 
>>> =
>> Filter...
>>> 
>>> Help!
>>> Thanks-
>>> Steve
>> 

RE: this nested foreach worked in 0.8.1, no longer works in 0.10

Posted by Steve Bernstein <St...@deem.com>.
Meanwhile, I'm moving forward running this script on 0.8.1 with no issues.  Probably going to back off on using 0.10 for now and wait for a more stable release.

-----Original Message-----
From: Steve Bernstein [mailto:Steve.Bernstein@deem.com] 
Sent: Sunday, June 03, 2012 9:25 AM
To: user@pig.apache.org
Subject: RE: this nested foreach worked in 0.8.1, no longer works in 0.10

Since this seems to do with project, before I wrote to the list, I'd already tried actionBag.sent and actionBagGrouped::actionBag.sent.  No joy.  Is this a bug?  Seems like others would have stumbled across it already.
SB
________________________________________
From: Subir S [subir.sasikumar@gmail.com]
Sent: Saturday, June 02, 2012 4:04 AM
To: user@pig.apache.org
Subject: Re: this nested foreach worked in 0.8.1, no longer works in 0.10

Can you try  sent = FILTER actionBag BY  actionBag.action=='sent';

Although not sure why it does not work. I also use pig 0.8.1 and similar constructs work for me.

On Fri, Jun 1, 2012 at 10:55 PM, Steve Bernstein
<St...@deem.com>wrote:

> No, all those relations embedded in the nested foreach are unique to 
> it, not used elsewhere.
> SB
>
> -----Original Message-----
> From: Dmitriy Ryaboy [mailto:dvryaboy@gmail.com]
> Sent: Friday, June 01, 2012 2:03 AM
> To: user@pig.apache.org
> Cc: user@pig.apache.org
> Subject: Re: this nested foreach worked in 0.8.1, no longer works in 
> 0.10
>
> Do you have relations ned sent, opened, etc, elsewhere in the script?
>
> On May 31, 2012, at 12:09 PM, Steve Bernstein 
> <St...@deem.com>
> wrote:
>
> > We upgraded from Pig 0.8.1 to 0.10 and the following nested foreach 
> > no
> longer works:
> >
> > actionBagGrouped = GROUP actionBag BY (deal_id,month); dealCounts = 
> > FOREACH actionBagGrouped {
> >                sent = FILTER actionBag BY action=='sent';
> >                opened = FILTER actionBag BY action=='opened';
> >                clicked = FILTER actionBag BY action=='clicked';
> >                purchased = FILTER actionBag BY action=='purchased';
> >
> >                GENERATE
> >                                FLATTEN(group),
> >                                COUNT(sent) AS sentCount,
> >                                COUNT(opened) AS openedCount,
> >                                COUNT(clicked) AS clickedCount,
> >                                COUNT(purchased) AS purchasedCount; 
> > }; I get:
> > "Invalid scalar projection: action : A column needs to be projected 
> > from
> a relation for it to be used as a scalar"
> > Banging my head against a wall-the problem is with the nest op sent 
> > =
> Filter...
> >
> > Help!
> > Thanks-
> > Steve
>

RE: this nested foreach worked in 0.8.1, no longer works in 0.10

Posted by Steve Bernstein <St...@deem.com>.
Since this seems to do with project, before I wrote to the list, I'd already tried actionBag.sent and actionBagGrouped::actionBag.sent.  No joy.  Is this a bug?  Seems like others would have stumbled across it already.
SB
________________________________________
From: Subir S [subir.sasikumar@gmail.com]
Sent: Saturday, June 02, 2012 4:04 AM
To: user@pig.apache.org
Subject: Re: this nested foreach worked in 0.8.1, no longer works in 0.10

Can you try  sent = FILTER actionBag BY  actionBag.action=='sent';

Although not sure why it does not work. I also use pig 0.8.1 and similar
constructs work for me.

On Fri, Jun 1, 2012 at 10:55 PM, Steve Bernstein
<St...@deem.com>wrote:

> No, all those relations embedded in the nested foreach are unique to it,
> not used elsewhere.
> SB
>
> -----Original Message-----
> From: Dmitriy Ryaboy [mailto:dvryaboy@gmail.com]
> Sent: Friday, June 01, 2012 2:03 AM
> To: user@pig.apache.org
> Cc: user@pig.apache.org
> Subject: Re: this nested foreach worked in 0.8.1, no longer works in 0.10
>
> Do you have relations ned sent, opened, etc, elsewhere in the script?
>
> On May 31, 2012, at 12:09 PM, Steve Bernstein <St...@deem.com>
> wrote:
>
> > We upgraded from Pig 0.8.1 to 0.10 and the following nested foreach no
> longer works:
> >
> > actionBagGrouped = GROUP actionBag BY (deal_id,month); dealCounts =
> > FOREACH actionBagGrouped {
> >                sent = FILTER actionBag BY action=='sent';
> >                opened = FILTER actionBag BY action=='opened';
> >                clicked = FILTER actionBag BY action=='clicked';
> >                purchased = FILTER actionBag BY action=='purchased';
> >
> >                GENERATE
> >                                FLATTEN(group),
> >                                COUNT(sent) AS sentCount,
> >                                COUNT(opened) AS openedCount,
> >                                COUNT(clicked) AS clickedCount,
> >                                COUNT(purchased) AS purchasedCount; };
> > I get:
> > "Invalid scalar projection: action : A column needs to be projected from
> a relation for it to be used as a scalar"
> > Banging my head against a wall-the problem is with the nest op sent =
> Filter...
> >
> > Help!
> > Thanks-
> > Steve
>

Re: this nested foreach worked in 0.8.1, no longer works in 0.10

Posted by Subir S <su...@gmail.com>.
Can you try  sent = FILTER actionBag BY  actionBag.action=='sent';

Although not sure why it does not work. I also use pig 0.8.1 and similar
constructs work for me.

On Fri, Jun 1, 2012 at 10:55 PM, Steve Bernstein
<St...@deem.com>wrote:

> No, all those relations embedded in the nested foreach are unique to it,
> not used elsewhere.
> SB
>
> -----Original Message-----
> From: Dmitriy Ryaboy [mailto:dvryaboy@gmail.com]
> Sent: Friday, June 01, 2012 2:03 AM
> To: user@pig.apache.org
> Cc: user@pig.apache.org
> Subject: Re: this nested foreach worked in 0.8.1, no longer works in 0.10
>
> Do you have relations ned sent, opened, etc, elsewhere in the script?
>
> On May 31, 2012, at 12:09 PM, Steve Bernstein <St...@deem.com>
> wrote:
>
> > We upgraded from Pig 0.8.1 to 0.10 and the following nested foreach no
> longer works:
> >
> > actionBagGrouped = GROUP actionBag BY (deal_id,month); dealCounts =
> > FOREACH actionBagGrouped {
> >                sent = FILTER actionBag BY action=='sent';
> >                opened = FILTER actionBag BY action=='opened';
> >                clicked = FILTER actionBag BY action=='clicked';
> >                purchased = FILTER actionBag BY action=='purchased';
> >
> >                GENERATE
> >                                FLATTEN(group),
> >                                COUNT(sent) AS sentCount,
> >                                COUNT(opened) AS openedCount,
> >                                COUNT(clicked) AS clickedCount,
> >                                COUNT(purchased) AS purchasedCount; };
> > I get:
> > "Invalid scalar projection: action : A column needs to be projected from
> a relation for it to be used as a scalar"
> > Banging my head against a wall-the problem is with the nest op sent =
> Filter...
> >
> > Help!
> > Thanks-
> > Steve
>

RE: this nested foreach worked in 0.8.1, no longer works in 0.10

Posted by Steve Bernstein <St...@deem.com>.
No, all those relations embedded in the nested foreach are unique to it, not used elsewhere.
SB

-----Original Message-----
From: Dmitriy Ryaboy [mailto:dvryaboy@gmail.com] 
Sent: Friday, June 01, 2012 2:03 AM
To: user@pig.apache.org
Cc: user@pig.apache.org
Subject: Re: this nested foreach worked in 0.8.1, no longer works in 0.10

Do you have relations ned sent, opened, etc, elsewhere in the script?

On May 31, 2012, at 12:09 PM, Steve Bernstein <St...@deem.com> wrote:

> We upgraded from Pig 0.8.1 to 0.10 and the following nested foreach no longer works:
> 
> actionBagGrouped = GROUP actionBag BY (deal_id,month); dealCounts = 
> FOREACH actionBagGrouped {
>                sent = FILTER actionBag BY action=='sent';
>                opened = FILTER actionBag BY action=='opened';
>                clicked = FILTER actionBag BY action=='clicked';
>                purchased = FILTER actionBag BY action=='purchased';
> 
>                GENERATE
>                                FLATTEN(group),
>                                COUNT(sent) AS sentCount,
>                                COUNT(opened) AS openedCount,
>                                COUNT(clicked) AS clickedCount,
>                                COUNT(purchased) AS purchasedCount; }; 
> I get:
> "Invalid scalar projection: action : A column needs to be projected from a relation for it to be used as a scalar"
> Banging my head against a wall-the problem is with the nest op sent = Filter...
> 
> Help!
> Thanks-
> Steve