You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Mridul Muralidharan <mr...@yahoo-inc.com> on 2008/03/27 23:31:24 UTC

BinCond related parse exceptions.

Hi,

   I am trying to execute the attached pig script - and run into parse 
exceptions.
Removing the outer BinCond does not result in parse errors ( cond == 3 ) 
- but is not the functionality I require : essentially, if all 
constraints are satisfied by some arbitrary set of tuples in A, then 
generate $1 else $0.

Thanks in advance for any pointers, suggestions or help.
Regards,
Mridul


A = load './a' USING PigStorage('\t');

B = GROUP A by $0;

C = FOREACH B {
   D1 = FILTER A by  ( $0 eq 'val1' ) ;
   D2 = FILTER A by  ( $1 eq 'val2' ) ;
   D3 = FILTER A by  ( $2 eq 'val3' ) ;
   GENERATE ( (  ( COUNT (D1) > 0 ? '1' : '0' ) + ( COUNT (D2) > 0 ? '1' 
: '0' ) + ( COUNT (D3) > 0 ? '1' : '0' )  ) == '3' ? $1 : $0 ) };

DUMP C;

Re: BinCond related parse exceptions.

Posted by Alan Gates <ga...@yahoo-inc.com>.
Rewriting your query as:
C = FOREACH B {
	D1 = FILTER A by  ( $0 eq 'val1' ) ;
	D2 = FILTER A by  ( $1 eq 'val2' ) ;
	D3 = FILTER A by  ( $2 eq 'val3' ) ;
	GENERATE (((COUNT(D1) > '0') and (COUNT(D2) > '0') and (COUNT(D3) > 
'0')) ? $1 : $0);
};

seems to work.  I'm not sure why it doesn't like it in the form you 
wrote it.

Alan.

On Mar 27, 2008, at 3:31 PM, Mridul Muralidharan wrote:

> Hi,
>
>   I am trying to execute the attached pig script - and run into parse 
> exceptions.
> Removing the outer BinCond does not result in parse errors ( cond == 3 
> ) - but is not the functionality I require : essentially, if all 
> constraints are satisfied by some arbitrary set of tuples in A, then 
> generate $1 else $0.
>
> Thanks in advance for any pointers, suggestions or help.
> Regards,
> Mridul
>
>
> A = load './a' USING PigStorage('\t');
>
> B = GROUP A by $0;
>
> C = FOREACH B {
>   D1 = FILTER A by  ( $0 eq 'val1' ) ;
>   D2 = FILTER A by  ( $1 eq 'val2' ) ;
>   D3 = FILTER A by  ( $2 eq 'val3' ) ;
>   GENERATE ( (  ( COUNT (D1) > 0 ? '1' : '0' ) + ( COUNT (D2) > 0 ? 
> '1' : '0' ) + ( COUNT (D3) > 0 ? '1' : '0' )  ) == '3' ? $1 : $0 ) };
>
> DUMP C;


Re: BinCond related parse exceptions.

Posted by Mridul Muralidharan <mr...@yahoo-inc.com>.
Hi,

   As part of translating my generated pig query to something 
presentable here, I missed out the quotes around the zero, the 
semi-colon and other syntax issues which were not really central to my 
problem.

On further analysis, what I found was that the actual problem seems to 
be that I cant specify two different tuple arity as part of the conditional.

What I mean is, this wont work :

-- start cut --

A = load './a' USING PigStorage('\t') AS (id, name, source_id, 
target_id, score, atomic_target_val);

B = A;

C = FILTER B by (  ( score > '0' ) and (  (  ( name eq 'Product_id'  and 
atomic_target_val eq '1'  and score > '0.5' )  )  or  (  ( name eq 
'type'  and atomic_target_val eq 'Product'  )  )  or  (  ( name eq 
'name'  )  )  )  );

D = GROUP C by source_id;

E = FOREACH D {
   F1 = FILTER C by  ( name eq 'Product_id'  and atomic_target_val eq 
'1'  and score > '0.5' ) ;
   F2 = FILTER C by  ( name eq 'type'  and atomic_target_val eq 
'Product'  ) ;
   GENERATE ( ( COUNT ( F1 ) > '0' ? '1' : '0' ) + ( COUNT ( F2 ) > '0' 
? '1' : '0' ) == '2' ? $1 : $0, $1) ; };

-- end cut --

or variants of it.

While this will :

   GENERATE ( ( COUNT ( F1 ) > '0' ? '1' : '0' ) + ( COUNT ( F2 ) > '0' 
? '1' : '0' ) == '2' ? '-1' : $0), $1;

The arity of the generated tuple is always 2 - $1 is pulled out of the 
bincond - maybe this is not (supposed to be) supported ...

This is not really what I am looking for and so need another step to 
retrieve what I need ... but it is something to move forward with !


Thanks,
Mridul



Utkarsh Srivastava wrote:
> I think you are missing a ";" before "}"
> 
> 
>> C = FOREACH B {
>>    D1 = FILTER A by  ( $0 eq 'val1' ) ;
>>    D2 = FILTER A by  ( $1 eq 'val2' ) ;
>>    D3 = FILTER A by  ( $2 eq 'val3' ) ;
>>    GENERATE ( (  ( COUNT (D1) > 0 ? '1' : '0' ) + ( COUNT (D2) > 0 ?
> '1'
>> : '0' ) + ( COUNT (D3) > 0 ? '1' : '0' )  ) == '3' ? $1 : $0 )
> <HERE!!> };
>> DUMP C;
> 
> 
> 


RE: BinCond related parse exceptions.

Posted by Utkarsh Srivastava <ut...@yahoo-inc.com>.
I think you are missing a ";" before "}"


> 
> C = FOREACH B {
>    D1 = FILTER A by  ( $0 eq 'val1' ) ;
>    D2 = FILTER A by  ( $1 eq 'val2' ) ;
>    D3 = FILTER A by  ( $2 eq 'val3' ) ;
>    GENERATE ( (  ( COUNT (D1) > 0 ? '1' : '0' ) + ( COUNT (D2) > 0 ?
'1'
> : '0' ) + ( COUNT (D3) > 0 ? '1' : '0' )  ) == '3' ? $1 : $0 )
<HERE!!> };
> 
> DUMP C;