You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Hitesh Patel <hi...@gmail.com> on 2013/12/26 17:33:06 UTC
Nested Binary Condition operators takes too much time to compile, PIG 0.11
Hello,
Below is the script which is working fine on PIG 0.8. Currently I am moving
from PIG 0.8 to 0.11. Trying to execute the same code on PIG 0.11. Its
taking too much time to compile the script.
A = LOAD 'data' USING PigStorage(',') AS
(code1:chararray,code2:chararray,code3:chararray);
B = FOREACH A GENERATE
(code1 MATCHES '(val1|val2|val3|val4|val5|val6|val7|val8|val9)'?
code1:'oth') AS code1,
(code2 MATCHES '(c1|c2|c3|v4|c5)'? code2: 'oth') AS code2,
code3;
C = FOREACH B GENERATE
(code2 == 'c1'? (code1 == 'val1' AND code3 =='xval' ? 1 : ( code1
== 'val1'? 2: ( code1 == 'val2'? 3: ( code1 == 'val3'? 4: ( code1 ==
'val4'? 5:( code1 == 'val5'? 6:( code1 == 'val6'? 7 : ( code1 == 'val7'? 8:
( code1 == 'val8'? 9 :10))))))))) :
(code2 == 'c2' ? (code1 == 'val1' AND code3 =='xval' ? 11 : ( code1
== 'val1'? 12: ( code1 == 'val2'? 13: ( code1 == 'val3'? 14: ( code1 ==
'val4'? 15:( code1 == 'val5'? 16:( code1 == 'val6'? 17 : 18))))))) :
(code2 == 'c3' ? (code1 =='val1' ? 19 : ( code1 == 'val2'? 20: (
code1 == 'val3'? 21: ( code1 == 'val4'? 22: ( code1 == 'val5'? 23:24))))):
(code2 == 'c4' ? (code1 =='val1' ? 25: ( code1 == 'val2'? 26: (
code1 == 'val3'? 27: ( code1 == 'val4'? 28: 29)))) :
(code1 =='val1' ? 30 : ( code1 == 'val2'? 31: ( code1 ==
'val3'? 32: ( code1 == 'val4'? 33: ( code1 == 'val5'? 34:35))))))))) as
code1,
(code2 == 'c1' ? 1: (code2== 'c2' ? 2 :( code2=='c3' ? 3 : (code2==
'c4'? 4:5)))) AS code2;
I am using Cloudera distribution. I tried same code on grunts for PIG 0.8
and PIG.11. On PIG 0.8 it compiles within a moment. On PIG 0.11 I executed
the line and waited almost for 12 mins, but control didn't come back grunt.
Do I need to set anything which I missed and having this problem?
--
Regards,
Hitesh Patel
Bardoli
Fix:- 91 2622 222299
Mob:- 91 942 835 7400
Re: Nested Binary Condition operators takes too much time to compile,
PIG 0.11
Posted by Cheolsoo Park <pi...@gmail.com>.
Hi Hitesh,
Sounds like you're hitting this bug-
https://issues.apache.org/jira/browse/PIG-2769
This is fixed in 0.12.0 (released) and 0.11.2 (not released).
Thanks,
Cheolsoo
On Thu, Dec 26, 2013 at 8:33 AM, Hitesh Patel <hi...@gmail.com>wrote:
> Hello,
>
> Below is the script which is working fine on PIG 0.8. Currently I am moving
> from PIG 0.8 to 0.11. Trying to execute the same code on PIG 0.11. Its
> taking too much time to compile the script.
>
> A = LOAD 'data' USING PigStorage(',') AS
> (code1:chararray,code2:chararray,code3:chararray);
> B = FOREACH A GENERATE
> (code1 MATCHES '(val1|val2|val3|val4|val5|val6|val7|val8|val9)'?
> code1:'oth') AS code1,
> (code2 MATCHES '(c1|c2|c3|v4|c5)'? code2: 'oth') AS code2,
> code3;
> C = FOREACH B GENERATE
> (code2 == 'c1'? (code1 == 'val1' AND code3 =='xval' ? 1 : ( code1
> == 'val1'? 2: ( code1 == 'val2'? 3: ( code1 == 'val3'? 4: ( code1 ==
> 'val4'? 5:( code1 == 'val5'? 6:( code1 == 'val6'? 7 : ( code1 == 'val7'? 8:
> ( code1 == 'val8'? 9 :10))))))))) :
> (code2 == 'c2' ? (code1 == 'val1' AND code3 =='xval' ? 11 : ( code1
> == 'val1'? 12: ( code1 == 'val2'? 13: ( code1 == 'val3'? 14: ( code1 ==
> 'val4'? 15:( code1 == 'val5'? 16:( code1 == 'val6'? 17 : 18))))))) :
> (code2 == 'c3' ? (code1 =='val1' ? 19 : ( code1 == 'val2'? 20: (
> code1 == 'val3'? 21: ( code1 == 'val4'? 22: ( code1 == 'val5'? 23:24))))):
> (code2 == 'c4' ? (code1 =='val1' ? 25: ( code1 == 'val2'? 26: (
> code1 == 'val3'? 27: ( code1 == 'val4'? 28: 29)))) :
> (code1 =='val1' ? 30 : ( code1 == 'val2'? 31: ( code1 ==
> 'val3'? 32: ( code1 == 'val4'? 33: ( code1 == 'val5'? 34:35))))))))) as
> code1,
> (code2 == 'c1' ? 1: (code2== 'c2' ? 2 :( code2=='c3' ? 3 : (code2==
> 'c4'? 4:5)))) AS code2;
>
> I am using Cloudera distribution. I tried same code on grunts for PIG 0.8
> and PIG.11. On PIG 0.8 it compiles within a moment. On PIG 0.11 I executed
> the line and waited almost for 12 mins, but control didn't come back grunt.
>
> Do I need to set anything which I missed and having this problem?
>
> --
> Regards,
> Hitesh Patel
> Bardoli
> Fix:- 91 2622 222299
> Mob:- 91 942 835 7400
>