You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Thejas M Nair (JIRA)" <ji...@apache.org> on 2010/10/07 22:31:34 UTC
[jira] Created: (PIG-1673) query with consecutive union-onschema
statement errors out
query with consecutive union-onschema statement errors out
----------------------------------------------------------
Key: PIG-1673
URL: https://issues.apache.org/jira/browse/PIG-1673
Project: Pig
Issue Type: Bug
Affects Versions: 0.8.0
Reporter: Thejas M Nair
Assignee: Thejas M Nair
Fix For: 0.8.0
grunt> l = load 't.txt' as (a,b);
grunt> f1 = foreach l generate (chararray)a;
grunt> u = union onschema f1, l;
grunt> u2 = union onschema u, f1;
2010-10-07 11:13:17,348 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1000: Error during parsing. null
Details at logfile: /Users/tejas/pig-0.8/branch-0.8/pig_1286475169418.log
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1673) query with consecutive union-onschema
statement errors out
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12922190#action_12922190 ]
Daniel Dai commented on PIG-1673:
---------------------------------
+1
> query with consecutive union-onschema statement errors out
> ----------------------------------------------------------
>
> Key: PIG-1673
> URL: https://issues.apache.org/jira/browse/PIG-1673
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Thejas M Nair
> Assignee: Thejas M Nair
> Fix For: 0.8.0
>
> Attachments: PIG-1673.1.patch
>
>
> grunt> l = load 't.txt' as (a,b);
> grunt> f1 = foreach l generate (chararray)a;
> grunt> u = union onschema f1, l;
> grunt> u2 = union onschema u, f1;
> 2010-10-07 11:13:17,348 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1000: Error during parsing. null
> Details at logfile: /Users/tejas/pig-0.8/branch-0.8/pig_1286475169418.log
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1673) query with consecutive union-onschema
statement errors out
Posted by "Thejas M Nair (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thejas M Nair updated PIG-1673:
-------------------------------
Attachment: PIG-1673.1.patch
The root cause for the issue was that the logical plan during parsing of a statement does not have the entire plan that has been generated at that point, and in case of union-onschema LogicalPlanValidator was being called with this partial plan.
When there were two consecutive union-onschema in the plan, code in TypeCheckingVisitor.insertCastForEachInBetweenIfNecessary assumes that predecessors of the firs LOUnion are available in the logical plan and it ends up throwing a NullPointerException .
Another issue caused by same root cause, was that the correct types would not be available in the parser at the point where union-onschema adds foreach statements and casts to the plan.
To address this problem, the patch moves most of the code for union-onschema in the parser to a visitor which is run from PigServer.compileLp(). The visitor has access to the the complete logical plan.
> query with consecutive union-onschema statement errors out
> ----------------------------------------------------------
>
> Key: PIG-1673
> URL: https://issues.apache.org/jira/browse/PIG-1673
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Thejas M Nair
> Assignee: Thejas M Nair
> Fix For: 0.8.0
>
> Attachments: PIG-1673.1.patch
>
>
> grunt> l = load 't.txt' as (a,b);
> grunt> f1 = foreach l generate (chararray)a;
> grunt> u = union onschema f1, l;
> grunt> u2 = union onschema u, f1;
> 2010-10-07 11:13:17,348 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1000: Error during parsing. null
> Details at logfile: /Users/tejas/pig-0.8/branch-0.8/pig_1286475169418.log
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1673) query with consecutive union-onschema
statement errors out
Posted by "Thejas M Nair (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thejas M Nair updated PIG-1673:
-------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
Patch committed to 0.8 branch and trunk.
> query with consecutive union-onschema statement errors out
> ----------------------------------------------------------
>
> Key: PIG-1673
> URL: https://issues.apache.org/jira/browse/PIG-1673
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Thejas M Nair
> Assignee: Thejas M Nair
> Fix For: 0.8.0
>
> Attachments: PIG-1673.1.patch
>
>
> grunt> l = load 't.txt' as (a,b);
> grunt> f1 = foreach l generate (chararray)a;
> grunt> u = union onschema f1, l;
> grunt> u2 = union onschema u, f1;
> 2010-10-07 11:13:17,348 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1000: Error during parsing. null
> Details at logfile: /Users/tejas/pig-0.8/branch-0.8/pig_1286475169418.log
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1673) query with consecutive union-onschema
statement errors out
Posted by "Thejas M Nair (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thejas M Nair updated PIG-1673:
-------------------------------
Status: Patch Available (was: Open)
Unit tests and test-patch have successfully completed.
> query with consecutive union-onschema statement errors out
> ----------------------------------------------------------
>
> Key: PIG-1673
> URL: https://issues.apache.org/jira/browse/PIG-1673
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Thejas M Nair
> Assignee: Thejas M Nair
> Fix For: 0.8.0
>
> Attachments: PIG-1673.1.patch
>
>
> grunt> l = load 't.txt' as (a,b);
> grunt> f1 = foreach l generate (chararray)a;
> grunt> u = union onschema f1, l;
> grunt> u2 = union onschema u, f1;
> 2010-10-07 11:13:17,348 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1000: Error during parsing. null
> Details at logfile: /Users/tejas/pig-0.8/branch-0.8/pig_1286475169418.log
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.