You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Vivek Padmanabhan (JIRA)" <ji...@apache.org> on 2011/02/18 13:38:38 UTC
[jira] Created: (PIG-1860) Bug in plan built for Nested foreach
Bug in plan built for Nested foreach
-------------------------------------
Key: PIG-1860
URL: https://issues.apache.org/jira/browse/PIG-1860
Project: Pig
Issue Type: Bug
Affects Versions: 0.8.0, 0.9.0
Reporter: Vivek Padmanabhan
Using the same inputs as in PIG-1858,
{code}
register myanotherudf.jar;
A = load 'myinput' using PigStorage() as ( date:chararray,bcookie:chararray,count:int,avg:double,pvs:int);
B = foreach A generate (int)(avg / 100.0) * 100 as avg, pvs;
C = group B by ( avg );
D = foreach C {
Pvs = order B by pvs;
Const = org.vivek.MyAnotherUDF(Pvs.pvs).(count,sum);
generate Const.sum as sum;
};
store D into 'out_D';
{code}
In this script even though I am passing Pvs.pvs to the UDF in the nested foreach, at runtime the "avg" is getting passed.
It looks like the logical plan created for D is wrong.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (PIG-1860) Bug in plan built for Nested foreach
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12996640#comment-12996640 ]
Olga Natkovich commented on PIG-1860:
-------------------------------------
I am ok with this patch as the short-term fix for 0.8 However, we need to revisit this for 0.9 there are a couple of things in our current approach that seems strange to me:
(1) That now we log temp jobs that failed but not the ones that succeed
(2) That reporting an error is so tightly connected with reporting stats for the store
> Bug in plan built for Nested foreach
> -------------------------------------
>
> Key: PIG-1860
> URL: https://issues.apache.org/jira/browse/PIG-1860
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.0, 0.9.0
> Reporter: Vivek Padmanabhan
>
> Using the same inputs as in PIG-1858,
> {code}
> register myanotherudf.jar;
> A = load 'myinput' using PigStorage() as ( date:chararray,bcookie:chararray,count:int,avg:double,pvs:int);
> B = foreach A generate (int)(avg / 100.0) * 100 as avg, pvs;
> C = group B by ( avg );
> D = foreach C {
> Pvs = order B by pvs;
> Const = org.vivek.MyAnotherUDF(Pvs.pvs).(count,sum);
> generate Const.sum as sum;
> };
> store D into 'out_D';
> {code}
> In this script even though I am passing Pvs.pvs to the UDF in the nested foreach, at runtime the "avg" is getting passed.
> It looks like the logical plan created for D is wrong.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (PIG-1860) Bug in plan built for Nested foreach
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai resolved PIG-1860.
-----------------------------
Resolution: Duplicate
The issue share the same problem with PIG-1858. Mark it as duplicate and move discussion to PIG-1858.
> Bug in plan built for Nested foreach
> -------------------------------------
>
> Key: PIG-1860
> URL: https://issues.apache.org/jira/browse/PIG-1860
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.0, 0.9.0
> Reporter: Vivek Padmanabhan
>
> Using the same inputs as in PIG-1858,
> {code}
> register myanotherudf.jar;
> A = load 'myinput' using PigStorage() as ( date:chararray,bcookie:chararray,count:int,avg:double,pvs:int);
> B = foreach A generate (int)(avg / 100.0) * 100 as avg, pvs;
> C = group B by ( avg );
> D = foreach C {
> Pvs = order B by pvs;
> Const = org.vivek.MyAnotherUDF(Pvs.pvs).(count,sum);
> generate Const.sum as sum;
> };
> store D into 'out_D';
> {code}
> In this script even though I am passing Pvs.pvs to the UDF in the nested foreach, at runtime the "avg" is getting passed.
> It looks like the logical plan created for D is wrong.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (PIG-1860) Bug in plan built for Nested foreach
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12996650#comment-12996650 ]
Olga Natkovich commented on PIG-1860:
-------------------------------------
the last comment was meant for a different issue - please ignore
> Bug in plan built for Nested foreach
> -------------------------------------
>
> Key: PIG-1860
> URL: https://issues.apache.org/jira/browse/PIG-1860
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.0, 0.9.0
> Reporter: Vivek Padmanabhan
>
> Using the same inputs as in PIG-1858,
> {code}
> register myanotherudf.jar;
> A = load 'myinput' using PigStorage() as ( date:chararray,bcookie:chararray,count:int,avg:double,pvs:int);
> B = foreach A generate (int)(avg / 100.0) * 100 as avg, pvs;
> C = group B by ( avg );
> D = foreach C {
> Pvs = order B by pvs;
> Const = org.vivek.MyAnotherUDF(Pvs.pvs).(count,sum);
> generate Const.sum as sum;
> };
> store D into 'out_D';
> {code}
> In this script even though I am passing Pvs.pvs to the UDF in the nested foreach, at runtime the "avg" is getting passed.
> It looks like the logical plan created for D is wrong.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira