You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Vivek Padmanabhan (Commented) (JIRA)" <ji...@apache.org> on 2011/10/17 13:10:10 UTC
[jira] [Commented] (PIG-2321) Pig 0.9 generates an improper MR plan
[ https://issues.apache.org/jira/browse/PIG-2321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13128789#comment-13128789 ]
Vivek Padmanabhan commented on PIG-2321:
----------------------------------------
The same script with 0.7 fails during the compile phase itself with the below message;
ERROR 1000: Error during parsing. Invalid alias: b2 in {group: (type: chararray,frcode: bytearray),a0: {frcode: bytearray,type: chararray,src_pty: chararray,srcpvid: chararray,src_spaceid: chararray,pageparams: bytearray}}
> Pig 0.9 generates an improper MR plan
> -------------------------------------
>
> Key: PIG-2321
> URL: https://issues.apache.org/jira/browse/PIG-2321
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.1, 0.9.0
> Reporter: Vivek Padmanabhan
> Priority: Minor
>
> The below script compiles and the MR plan has 3 jobs. But the script will always fail.
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 3
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 2
> {code}
> a0 = load 'i1' as (frcode,type:chararray,src_pty:chararray,srcpvid:chararray,src_spaceid:chararray,pageparams);
> b2 = group a0 by (type, frcode) parallel 2;
> b3 = foreach b2 generate COUNT_STAR(b2.$0) as num_clicks,flatten(group) ;
> store b3 into 'junkout3';
> {code}
> The script fails due to improper reference at COUNT_STAR(b2.$0) . But this validation should be done at the compile time.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] [Commented] (PIG-2321) Pig 0.9 generates an improper MR plan
Posted by Dmitriy Ryaboy <dv...@gmail.com>.
Vivek, that issue has been reported -- it's pig thinking you are trying to use the new (as of 0.8) scalar feature.
On Oct 17, 2011, at 4:10 AM, "Vivek Padmanabhan (Commented) (JIRA)" <ji...@apache.org> wrote:
>
> [ https://issues.apache.org/jira/browse/PIG-2321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13128789#comment-13128789 ]
>
> Vivek Padmanabhan commented on PIG-2321:
> ----------------------------------------
>
> The same script with 0.7 fails during the compile phase itself with the below message;
>
> ERROR 1000: Error during parsing. Invalid alias: b2 in {group: (type: chararray,frcode: bytearray),a0: {frcode: bytearray,type: chararray,src_pty: chararray,srcpvid: chararray,src_spaceid: chararray,pageparams: bytearray}}
>
>> Pig 0.9 generates an improper MR plan
>> -------------------------------------
>>
>> Key: PIG-2321
>> URL: https://issues.apache.org/jira/browse/PIG-2321
>> Project: Pig
>> Issue Type: Bug
>> Affects Versions: 0.8.1, 0.9.0
>> Reporter: Vivek Padmanabhan
>> Priority: Minor
>>
>> The below script compiles and the MR plan has 3 jobs. But the script will always fail.
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 3
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 2
>> {code}
>> a0 = load 'i1' as (frcode,type:chararray,src_pty:chararray,srcpvid:chararray,src_spaceid:chararray,pageparams);
>> b2 = group a0 by (type, frcode) parallel 2;
>> b3 = foreach b2 generate COUNT_STAR(b2.$0) as num_clicks,flatten(group) ;
>> store b3 into 'junkout3';
>> {code}
>> The script fails due to improper reference at COUNT_STAR(b2.$0) . But this validation should be done at the compile time.
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>
>