You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Mridul Muralidharan <mr...@yahoo-inc.com> on 2008/03/27 23:31:24 UTC
BinCond related parse exceptions.
Hi,
I am trying to execute the attached pig script - and run into parse
exceptions.
Removing the outer BinCond does not result in parse errors ( cond == 3 )
- but is not the functionality I require : essentially, if all
constraints are satisfied by some arbitrary set of tuples in A, then
generate $1 else $0.
Thanks in advance for any pointers, suggestions or help.
Regards,
Mridul
A = load './a' USING PigStorage('\t');
B = GROUP A by $0;
C = FOREACH B {
D1 = FILTER A by ( $0 eq 'val1' ) ;
D2 = FILTER A by ( $1 eq 'val2' ) ;
D3 = FILTER A by ( $2 eq 'val3' ) ;
GENERATE ( ( ( COUNT (D1) > 0 ? '1' : '0' ) + ( COUNT (D2) > 0 ? '1'
: '0' ) + ( COUNT (D3) > 0 ? '1' : '0' ) ) == '3' ? $1 : $0 ) };
DUMP C;
Re: BinCond related parse exceptions.
Posted by Alan Gates <ga...@yahoo-inc.com>.
Rewriting your query as:
C = FOREACH B {
D1 = FILTER A by ( $0 eq 'val1' ) ;
D2 = FILTER A by ( $1 eq 'val2' ) ;
D3 = FILTER A by ( $2 eq 'val3' ) ;
GENERATE (((COUNT(D1) > '0') and (COUNT(D2) > '0') and (COUNT(D3) >
'0')) ? $1 : $0);
};
seems to work. I'm not sure why it doesn't like it in the form you
wrote it.
Alan.
On Mar 27, 2008, at 3:31 PM, Mridul Muralidharan wrote:
> Hi,
>
> I am trying to execute the attached pig script - and run into parse
> exceptions.
> Removing the outer BinCond does not result in parse errors ( cond == 3
> ) - but is not the functionality I require : essentially, if all
> constraints are satisfied by some arbitrary set of tuples in A, then
> generate $1 else $0.
>
> Thanks in advance for any pointers, suggestions or help.
> Regards,
> Mridul
>
>
> A = load './a' USING PigStorage('\t');
>
> B = GROUP A by $0;
>
> C = FOREACH B {
> D1 = FILTER A by ( $0 eq 'val1' ) ;
> D2 = FILTER A by ( $1 eq 'val2' ) ;
> D3 = FILTER A by ( $2 eq 'val3' ) ;
> GENERATE ( ( ( COUNT (D1) > 0 ? '1' : '0' ) + ( COUNT (D2) > 0 ?
> '1' : '0' ) + ( COUNT (D3) > 0 ? '1' : '0' ) ) == '3' ? $1 : $0 ) };
>
> DUMP C;
Re: BinCond related parse exceptions.
Posted by Mridul Muralidharan <mr...@yahoo-inc.com>.
Hi,
As part of translating my generated pig query to something
presentable here, I missed out the quotes around the zero, the
semi-colon and other syntax issues which were not really central to my
problem.
On further analysis, what I found was that the actual problem seems to
be that I cant specify two different tuple arity as part of the conditional.
What I mean is, this wont work :
-- start cut --
A = load './a' USING PigStorage('\t') AS (id, name, source_id,
target_id, score, atomic_target_val);
B = A;
C = FILTER B by ( ( score > '0' ) and ( ( ( name eq 'Product_id' and
atomic_target_val eq '1' and score > '0.5' ) ) or ( ( name eq
'type' and atomic_target_val eq 'Product' ) ) or ( ( name eq
'name' ) ) ) );
D = GROUP C by source_id;
E = FOREACH D {
F1 = FILTER C by ( name eq 'Product_id' and atomic_target_val eq
'1' and score > '0.5' ) ;
F2 = FILTER C by ( name eq 'type' and atomic_target_val eq
'Product' ) ;
GENERATE ( ( COUNT ( F1 ) > '0' ? '1' : '0' ) + ( COUNT ( F2 ) > '0'
? '1' : '0' ) == '2' ? $1 : $0, $1) ; };
-- end cut --
or variants of it.
While this will :
GENERATE ( ( COUNT ( F1 ) > '0' ? '1' : '0' ) + ( COUNT ( F2 ) > '0'
? '1' : '0' ) == '2' ? '-1' : $0), $1;
The arity of the generated tuple is always 2 - $1 is pulled out of the
bincond - maybe this is not (supposed to be) supported ...
This is not really what I am looking for and so need another step to
retrieve what I need ... but it is something to move forward with !
Thanks,
Mridul
Utkarsh Srivastava wrote:
> I think you are missing a ";" before "}"
>
>
>> C = FOREACH B {
>> D1 = FILTER A by ( $0 eq 'val1' ) ;
>> D2 = FILTER A by ( $1 eq 'val2' ) ;
>> D3 = FILTER A by ( $2 eq 'val3' ) ;
>> GENERATE ( ( ( COUNT (D1) > 0 ? '1' : '0' ) + ( COUNT (D2) > 0 ?
> '1'
>> : '0' ) + ( COUNT (D3) > 0 ? '1' : '0' ) ) == '3' ? $1 : $0 )
> <HERE!!> };
>> DUMP C;
>
>
>
RE: BinCond related parse exceptions.
Posted by Utkarsh Srivastava <ut...@yahoo-inc.com>.
I think you are missing a ";" before "}"
>
> C = FOREACH B {
> D1 = FILTER A by ( $0 eq 'val1' ) ;
> D2 = FILTER A by ( $1 eq 'val2' ) ;
> D3 = FILTER A by ( $2 eq 'val3' ) ;
> GENERATE ( ( ( COUNT (D1) > 0 ? '1' : '0' ) + ( COUNT (D2) > 0 ?
'1'
> : '0' ) + ( COUNT (D3) > 0 ? '1' : '0' ) ) == '3' ? $1 : $0 )
<HERE!!> };
>
> DUMP C;