You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Ankur (JIRA)" <ji...@apache.org> on 2009/11/06 08:02:41 UTC
[jira] Created: (PIG-1075) Error in Cogroup when key fields types
don't match
Error in Cogroup when key fields types don't match
--------------------------------------------------
Key: PIG-1075
URL: https://issues.apache.org/jira/browse/PIG-1075
Project: Pig
Issue Type: Bug
Affects Versions: 0.5.0
Reporter: Ankur
When Cogrouping 2 relations on multiple key fields, pig throws an error if the corresponding types don't match.
Consider the following script:-
A = LOAD 'data' USING PigStorage() as (a:chararray, b:int, c:int);
B = LOAD 'data' USING PigStorage() as (a:chararray, b:chararray, c:int);
C = CoGROUP A BY (a,b,c), B BY (a,b,c);
D = FOREACH C GENERATE FLATTEN(A), FLATTEN(B);
describe D;
dump D;
The complete stack trace of the error thrown is
Pig Stack Trace
---------------
ERROR 1051: Cannot cast to Unknown
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1001: Unable to describe schema for alias D
at org.apache.pig.PigServer.dumpSchema(PigServer.java:436)
at org.apache.pig.tools.grunt.GruntParser.processDescribe(GruntParser.java:233)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:253)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:144)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89)
at org.apache.pig.Main.main(Main.java:397)
Caused by: org.apache.pig.impl.plan.PlanValidationException: ERROR 0: An unexpected exception caused the validation to stop
at org.apache.pig.impl.plan.PlanValidator.validateSkipCollectException(PlanValidator.java:104)
at org.apache.pig.impl.logicalLayer.validators.TypeCheckingValidator.validate(TypeCheckingValidator.java:40)
at org.apache.pig.impl.logicalLayer.validators.TypeCheckingValidator.validate(TypeCheckingValidator.java:30)
at org.apache.pig.impl.logicalLayer.validators.LogicalPlanValidationExecutor.validate(LogicalPlanValidationExecutor.java:83)
at org.apache.pig.PigServer.compileLp(PigServer.java:821)
at org.apache.pig.PigServer.dumpSchema(PigServer.java:428)
... 6 more
Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: ERROR 1060: Cannot resolve COGroup output schema
at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.visit(TypeCheckingVisitor.java:2463)
at org.apache.pig.impl.logicalLayer.LOCogroup.visit(LOCogroup.java:372)
at org.apache.pig.impl.logicalLayer.LOCogroup.visit(LOCogroup.java:45)
at org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:69)
at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
at org.apache.pig.impl.plan.PlanValidator.validateSkipCollectException(PlanValidator.java:101)
... 11 more
Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: ERROR 1051: Cannot cast to Unknown
at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.insertAtomicCastForCOGroupInnerPlan(TypeCheckingVisitor.java:2552)
at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.visit(TypeCheckingVisitor.java:2451)
... 16 more
The error message does not help the user in identifying the issue clearly especially if the pig script is large and complex.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1075) Error in Cogroup when key fields types
don't match
Posted by "Richard Ding (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Richard Ding updated PIG-1075:
------------------------------
Attachment: PIG-1075.patch
This patch moves the error up to the parser and gives a better error message for cogroup statement with incompatible group types:
Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1110: Cogroup column 1 has incompatible types: chararray versus int
at org.apache.pig.impl.logicalLayer.LOCogroup.getTupleGroupBySchema(LOCogroup.java:499)
at org.apache.pig.impl.logicalLayer.LOCogroup.getSchema(LOCogroup.java:325)
at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:779)
> Error in Cogroup when key fields types don't match
> --------------------------------------------------
>
> Key: PIG-1075
> URL: https://issues.apache.org/jira/browse/PIG-1075
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.5.0
> Reporter: Ankur
> Attachments: PIG-1075.patch
>
>
> When Cogrouping 2 relations on multiple key fields, pig throws an error if the corresponding types don't match.
> Consider the following script:-
> A = LOAD 'data' USING PigStorage() as (a:chararray, b:int, c:int);
> B = LOAD 'data' USING PigStorage() as (a:chararray, b:chararray, c:int);
> C = CoGROUP A BY (a,b,c), B BY (a,b,c);
> D = FOREACH C GENERATE FLATTEN(A), FLATTEN(B);
> describe D;
> dump D;
> The complete stack trace of the error thrown is
> Pig Stack Trace
> ---------------
> ERROR 1051: Cannot cast to Unknown
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1001: Unable to describe schema for alias D
> at org.apache.pig.PigServer.dumpSchema(PigServer.java:436)
> at org.apache.pig.tools.grunt.GruntParser.processDescribe(GruntParser.java:233)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:253)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:144)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89)
> at org.apache.pig.Main.main(Main.java:397)
> Caused by: org.apache.pig.impl.plan.PlanValidationException: ERROR 0: An unexpected exception caused the validation to stop
> at org.apache.pig.impl.plan.PlanValidator.validateSkipCollectException(PlanValidator.java:104)
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingValidator.validate(TypeCheckingValidator.java:40)
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingValidator.validate(TypeCheckingValidator.java:30)
> at org.apache.pig.impl.logicalLayer.validators.LogicalPlanValidationExecutor.validate(LogicalPlanValidationExecutor.java:83)
> at org.apache.pig.PigServer.compileLp(PigServer.java:821)
> at org.apache.pig.PigServer.dumpSchema(PigServer.java:428)
> ... 6 more
> Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: ERROR 1060: Cannot resolve COGroup output schema
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.visit(TypeCheckingVisitor.java:2463)
> at org.apache.pig.impl.logicalLayer.LOCogroup.visit(LOCogroup.java:372)
> at org.apache.pig.impl.logicalLayer.LOCogroup.visit(LOCogroup.java:45)
> at org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:69)
> at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
> at org.apache.pig.impl.plan.PlanValidator.validateSkipCollectException(PlanValidator.java:101)
> ... 11 more
> Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: ERROR 1051: Cannot cast to Unknown
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.insertAtomicCastForCOGroupInnerPlan(TypeCheckingVisitor.java:2552)
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.visit(TypeCheckingVisitor.java:2451)
> ... 16 more
> The error message does not help the user in identifying the issue clearly especially if the pig script is large and complex.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Closed: (PIG-1075) Error in Cogroup when key fields types
don't match
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai closed PIG-1075.
---------------------------
> Error in Cogroup when key fields types don't match
> --------------------------------------------------
>
> Key: PIG-1075
> URL: https://issues.apache.org/jira/browse/PIG-1075
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.5.0
> Reporter: Ankur
> Assignee: Richard Ding
> Fix For: 0.7.0
>
> Attachments: PIG-1075.patch
>
>
> When Cogrouping 2 relations on multiple key fields, pig throws an error if the corresponding types don't match.
> Consider the following script:-
> A = LOAD 'data' USING PigStorage() as (a:chararray, b:int, c:int);
> B = LOAD 'data' USING PigStorage() as (a:chararray, b:chararray, c:int);
> C = CoGROUP A BY (a,b,c), B BY (a,b,c);
> D = FOREACH C GENERATE FLATTEN(A), FLATTEN(B);
> describe D;
> dump D;
> The complete stack trace of the error thrown is
> Pig Stack Trace
> ---------------
> ERROR 1051: Cannot cast to Unknown
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1001: Unable to describe schema for alias D
> at org.apache.pig.PigServer.dumpSchema(PigServer.java:436)
> at org.apache.pig.tools.grunt.GruntParser.processDescribe(GruntParser.java:233)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:253)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:144)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89)
> at org.apache.pig.Main.main(Main.java:397)
> Caused by: org.apache.pig.impl.plan.PlanValidationException: ERROR 0: An unexpected exception caused the validation to stop
> at org.apache.pig.impl.plan.PlanValidator.validateSkipCollectException(PlanValidator.java:104)
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingValidator.validate(TypeCheckingValidator.java:40)
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingValidator.validate(TypeCheckingValidator.java:30)
> at org.apache.pig.impl.logicalLayer.validators.LogicalPlanValidationExecutor.validate(LogicalPlanValidationExecutor.java:83)
> at org.apache.pig.PigServer.compileLp(PigServer.java:821)
> at org.apache.pig.PigServer.dumpSchema(PigServer.java:428)
> ... 6 more
> Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: ERROR 1060: Cannot resolve COGroup output schema
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.visit(TypeCheckingVisitor.java:2463)
> at org.apache.pig.impl.logicalLayer.LOCogroup.visit(LOCogroup.java:372)
> at org.apache.pig.impl.logicalLayer.LOCogroup.visit(LOCogroup.java:45)
> at org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:69)
> at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
> at org.apache.pig.impl.plan.PlanValidator.validateSkipCollectException(PlanValidator.java:101)
> ... 11 more
> Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: ERROR 1051: Cannot cast to Unknown
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.insertAtomicCastForCOGroupInnerPlan(TypeCheckingVisitor.java:2552)
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.visit(TypeCheckingVisitor.java:2451)
> ... 16 more
> The error message does not help the user in identifying the issue clearly especially if the pig script is large and complex.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1075) Error in Cogroup when key fields types
don't match
Posted by "Ankur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12774222#action_12774222 ]
Ankur commented on PIG-1075:
----------------------------
Pig should throw an error message that better identifies the cause of the problem.
> Error in Cogroup when key fields types don't match
> --------------------------------------------------
>
> Key: PIG-1075
> URL: https://issues.apache.org/jira/browse/PIG-1075
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.5.0
> Reporter: Ankur
>
> When Cogrouping 2 relations on multiple key fields, pig throws an error if the corresponding types don't match.
> Consider the following script:-
> A = LOAD 'data' USING PigStorage() as (a:chararray, b:int, c:int);
> B = LOAD 'data' USING PigStorage() as (a:chararray, b:chararray, c:int);
> C = CoGROUP A BY (a,b,c), B BY (a,b,c);
> D = FOREACH C GENERATE FLATTEN(A), FLATTEN(B);
> describe D;
> dump D;
> The complete stack trace of the error thrown is
> Pig Stack Trace
> ---------------
> ERROR 1051: Cannot cast to Unknown
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1001: Unable to describe schema for alias D
> at org.apache.pig.PigServer.dumpSchema(PigServer.java:436)
> at org.apache.pig.tools.grunt.GruntParser.processDescribe(GruntParser.java:233)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:253)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:144)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89)
> at org.apache.pig.Main.main(Main.java:397)
> Caused by: org.apache.pig.impl.plan.PlanValidationException: ERROR 0: An unexpected exception caused the validation to stop
> at org.apache.pig.impl.plan.PlanValidator.validateSkipCollectException(PlanValidator.java:104)
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingValidator.validate(TypeCheckingValidator.java:40)
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingValidator.validate(TypeCheckingValidator.java:30)
> at org.apache.pig.impl.logicalLayer.validators.LogicalPlanValidationExecutor.validate(LogicalPlanValidationExecutor.java:83)
> at org.apache.pig.PigServer.compileLp(PigServer.java:821)
> at org.apache.pig.PigServer.dumpSchema(PigServer.java:428)
> ... 6 more
> Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: ERROR 1060: Cannot resolve COGroup output schema
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.visit(TypeCheckingVisitor.java:2463)
> at org.apache.pig.impl.logicalLayer.LOCogroup.visit(LOCogroup.java:372)
> at org.apache.pig.impl.logicalLayer.LOCogroup.visit(LOCogroup.java:45)
> at org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:69)
> at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
> at org.apache.pig.impl.plan.PlanValidator.validateSkipCollectException(PlanValidator.java:101)
> ... 11 more
> Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: ERROR 1051: Cannot cast to Unknown
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.insertAtomicCastForCOGroupInnerPlan(TypeCheckingVisitor.java:2552)
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.visit(TypeCheckingVisitor.java:2451)
> ... 16 more
> The error message does not help the user in identifying the issue clearly especially if the pig script is large and complex.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1075) Error in Cogroup when key fields types
don't match
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich updated PIG-1075:
--------------------------------
Fix Version/s: 0.7.0
> Error in Cogroup when key fields types don't match
> --------------------------------------------------
>
> Key: PIG-1075
> URL: https://issues.apache.org/jira/browse/PIG-1075
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.5.0
> Reporter: Ankur
> Assignee: Richard Ding
> Fix For: 0.7.0
>
> Attachments: PIG-1075.patch
>
>
> When Cogrouping 2 relations on multiple key fields, pig throws an error if the corresponding types don't match.
> Consider the following script:-
> A = LOAD 'data' USING PigStorage() as (a:chararray, b:int, c:int);
> B = LOAD 'data' USING PigStorage() as (a:chararray, b:chararray, c:int);
> C = CoGROUP A BY (a,b,c), B BY (a,b,c);
> D = FOREACH C GENERATE FLATTEN(A), FLATTEN(B);
> describe D;
> dump D;
> The complete stack trace of the error thrown is
> Pig Stack Trace
> ---------------
> ERROR 1051: Cannot cast to Unknown
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1001: Unable to describe schema for alias D
> at org.apache.pig.PigServer.dumpSchema(PigServer.java:436)
> at org.apache.pig.tools.grunt.GruntParser.processDescribe(GruntParser.java:233)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:253)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:144)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89)
> at org.apache.pig.Main.main(Main.java:397)
> Caused by: org.apache.pig.impl.plan.PlanValidationException: ERROR 0: An unexpected exception caused the validation to stop
> at org.apache.pig.impl.plan.PlanValidator.validateSkipCollectException(PlanValidator.java:104)
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingValidator.validate(TypeCheckingValidator.java:40)
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingValidator.validate(TypeCheckingValidator.java:30)
> at org.apache.pig.impl.logicalLayer.validators.LogicalPlanValidationExecutor.validate(LogicalPlanValidationExecutor.java:83)
> at org.apache.pig.PigServer.compileLp(PigServer.java:821)
> at org.apache.pig.PigServer.dumpSchema(PigServer.java:428)
> ... 6 more
> Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: ERROR 1060: Cannot resolve COGroup output schema
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.visit(TypeCheckingVisitor.java:2463)
> at org.apache.pig.impl.logicalLayer.LOCogroup.visit(LOCogroup.java:372)
> at org.apache.pig.impl.logicalLayer.LOCogroup.visit(LOCogroup.java:45)
> at org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:69)
> at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
> at org.apache.pig.impl.plan.PlanValidator.validateSkipCollectException(PlanValidator.java:101)
> ... 11 more
> Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: ERROR 1051: Cannot cast to Unknown
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.insertAtomicCastForCOGroupInnerPlan(TypeCheckingVisitor.java:2552)
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.visit(TypeCheckingVisitor.java:2451)
> ... 16 more
> The error message does not help the user in identifying the issue clearly especially if the pig script is large and complex.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (PIG-1075) Error in Cogroup when key fields types
don't match
Posted by "Richard Ding (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Richard Ding reassigned PIG-1075:
---------------------------------
Assignee: Richard Ding
> Error in Cogroup when key fields types don't match
> --------------------------------------------------
>
> Key: PIG-1075
> URL: https://issues.apache.org/jira/browse/PIG-1075
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.5.0
> Reporter: Ankur
> Assignee: Richard Ding
> Attachments: PIG-1075.patch
>
>
> When Cogrouping 2 relations on multiple key fields, pig throws an error if the corresponding types don't match.
> Consider the following script:-
> A = LOAD 'data' USING PigStorage() as (a:chararray, b:int, c:int);
> B = LOAD 'data' USING PigStorage() as (a:chararray, b:chararray, c:int);
> C = CoGROUP A BY (a,b,c), B BY (a,b,c);
> D = FOREACH C GENERATE FLATTEN(A), FLATTEN(B);
> describe D;
> dump D;
> The complete stack trace of the error thrown is
> Pig Stack Trace
> ---------------
> ERROR 1051: Cannot cast to Unknown
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1001: Unable to describe schema for alias D
> at org.apache.pig.PigServer.dumpSchema(PigServer.java:436)
> at org.apache.pig.tools.grunt.GruntParser.processDescribe(GruntParser.java:233)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:253)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:144)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89)
> at org.apache.pig.Main.main(Main.java:397)
> Caused by: org.apache.pig.impl.plan.PlanValidationException: ERROR 0: An unexpected exception caused the validation to stop
> at org.apache.pig.impl.plan.PlanValidator.validateSkipCollectException(PlanValidator.java:104)
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingValidator.validate(TypeCheckingValidator.java:40)
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingValidator.validate(TypeCheckingValidator.java:30)
> at org.apache.pig.impl.logicalLayer.validators.LogicalPlanValidationExecutor.validate(LogicalPlanValidationExecutor.java:83)
> at org.apache.pig.PigServer.compileLp(PigServer.java:821)
> at org.apache.pig.PigServer.dumpSchema(PigServer.java:428)
> ... 6 more
> Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: ERROR 1060: Cannot resolve COGroup output schema
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.visit(TypeCheckingVisitor.java:2463)
> at org.apache.pig.impl.logicalLayer.LOCogroup.visit(LOCogroup.java:372)
> at org.apache.pig.impl.logicalLayer.LOCogroup.visit(LOCogroup.java:45)
> at org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:69)
> at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
> at org.apache.pig.impl.plan.PlanValidator.validateSkipCollectException(PlanValidator.java:101)
> ... 11 more
> Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: ERROR 1051: Cannot cast to Unknown
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.insertAtomicCastForCOGroupInnerPlan(TypeCheckingVisitor.java:2552)
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.visit(TypeCheckingVisitor.java:2451)
> ... 16 more
> The error message does not help the user in identifying the issue clearly especially if the pig script is large and complex.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1075) Error in Cogroup when key fields types
don't match
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12790951#action_12790951 ]
Hadoop QA commented on PIG-1075:
--------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12427991/PIG-1075.patch
against trunk revision 890596.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/124/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/124/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/124/console
This message is automatically generated.
> Error in Cogroup when key fields types don't match
> --------------------------------------------------
>
> Key: PIG-1075
> URL: https://issues.apache.org/jira/browse/PIG-1075
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.5.0
> Reporter: Ankur
> Assignee: Richard Ding
> Attachments: PIG-1075.patch
>
>
> When Cogrouping 2 relations on multiple key fields, pig throws an error if the corresponding types don't match.
> Consider the following script:-
> A = LOAD 'data' USING PigStorage() as (a:chararray, b:int, c:int);
> B = LOAD 'data' USING PigStorage() as (a:chararray, b:chararray, c:int);
> C = CoGROUP A BY (a,b,c), B BY (a,b,c);
> D = FOREACH C GENERATE FLATTEN(A), FLATTEN(B);
> describe D;
> dump D;
> The complete stack trace of the error thrown is
> Pig Stack Trace
> ---------------
> ERROR 1051: Cannot cast to Unknown
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1001: Unable to describe schema for alias D
> at org.apache.pig.PigServer.dumpSchema(PigServer.java:436)
> at org.apache.pig.tools.grunt.GruntParser.processDescribe(GruntParser.java:233)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:253)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:144)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89)
> at org.apache.pig.Main.main(Main.java:397)
> Caused by: org.apache.pig.impl.plan.PlanValidationException: ERROR 0: An unexpected exception caused the validation to stop
> at org.apache.pig.impl.plan.PlanValidator.validateSkipCollectException(PlanValidator.java:104)
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingValidator.validate(TypeCheckingValidator.java:40)
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingValidator.validate(TypeCheckingValidator.java:30)
> at org.apache.pig.impl.logicalLayer.validators.LogicalPlanValidationExecutor.validate(LogicalPlanValidationExecutor.java:83)
> at org.apache.pig.PigServer.compileLp(PigServer.java:821)
> at org.apache.pig.PigServer.dumpSchema(PigServer.java:428)
> ... 6 more
> Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: ERROR 1060: Cannot resolve COGroup output schema
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.visit(TypeCheckingVisitor.java:2463)
> at org.apache.pig.impl.logicalLayer.LOCogroup.visit(LOCogroup.java:372)
> at org.apache.pig.impl.logicalLayer.LOCogroup.visit(LOCogroup.java:45)
> at org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:69)
> at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
> at org.apache.pig.impl.plan.PlanValidator.validateSkipCollectException(PlanValidator.java:101)
> ... 11 more
> Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: ERROR 1051: Cannot cast to Unknown
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.insertAtomicCastForCOGroupInnerPlan(TypeCheckingVisitor.java:2552)
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.visit(TypeCheckingVisitor.java:2451)
> ... 16 more
> The error message does not help the user in identifying the issue clearly especially if the pig script is large and complex.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1075) Error in Cogroup when key fields types
don't match
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12791689#action_12791689 ]
Olga Natkovich commented on PIG-1075:
-------------------------------------
+1 on the change. will be committing it shortly.
> Error in Cogroup when key fields types don't match
> --------------------------------------------------
>
> Key: PIG-1075
> URL: https://issues.apache.org/jira/browse/PIG-1075
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.5.0
> Reporter: Ankur
> Assignee: Richard Ding
> Fix For: 0.7.0
>
> Attachments: PIG-1075.patch
>
>
> When Cogrouping 2 relations on multiple key fields, pig throws an error if the corresponding types don't match.
> Consider the following script:-
> A = LOAD 'data' USING PigStorage() as (a:chararray, b:int, c:int);
> B = LOAD 'data' USING PigStorage() as (a:chararray, b:chararray, c:int);
> C = CoGROUP A BY (a,b,c), B BY (a,b,c);
> D = FOREACH C GENERATE FLATTEN(A), FLATTEN(B);
> describe D;
> dump D;
> The complete stack trace of the error thrown is
> Pig Stack Trace
> ---------------
> ERROR 1051: Cannot cast to Unknown
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1001: Unable to describe schema for alias D
> at org.apache.pig.PigServer.dumpSchema(PigServer.java:436)
> at org.apache.pig.tools.grunt.GruntParser.processDescribe(GruntParser.java:233)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:253)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:144)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89)
> at org.apache.pig.Main.main(Main.java:397)
> Caused by: org.apache.pig.impl.plan.PlanValidationException: ERROR 0: An unexpected exception caused the validation to stop
> at org.apache.pig.impl.plan.PlanValidator.validateSkipCollectException(PlanValidator.java:104)
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingValidator.validate(TypeCheckingValidator.java:40)
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingValidator.validate(TypeCheckingValidator.java:30)
> at org.apache.pig.impl.logicalLayer.validators.LogicalPlanValidationExecutor.validate(LogicalPlanValidationExecutor.java:83)
> at org.apache.pig.PigServer.compileLp(PigServer.java:821)
> at org.apache.pig.PigServer.dumpSchema(PigServer.java:428)
> ... 6 more
> Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: ERROR 1060: Cannot resolve COGroup output schema
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.visit(TypeCheckingVisitor.java:2463)
> at org.apache.pig.impl.logicalLayer.LOCogroup.visit(LOCogroup.java:372)
> at org.apache.pig.impl.logicalLayer.LOCogroup.visit(LOCogroup.java:45)
> at org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:69)
> at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
> at org.apache.pig.impl.plan.PlanValidator.validateSkipCollectException(PlanValidator.java:101)
> ... 11 more
> Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: ERROR 1051: Cannot cast to Unknown
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.insertAtomicCastForCOGroupInnerPlan(TypeCheckingVisitor.java:2552)
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.visit(TypeCheckingVisitor.java:2451)
> ... 16 more
> The error message does not help the user in identifying the issue clearly especially if the pig script is large and complex.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1075) Error in Cogroup when key fields types
don't match
Posted by "Richard Ding (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Richard Ding updated PIG-1075:
------------------------------
Status: Patch Available (was: Open)
> Error in Cogroup when key fields types don't match
> --------------------------------------------------
>
> Key: PIG-1075
> URL: https://issues.apache.org/jira/browse/PIG-1075
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.5.0
> Reporter: Ankur
> Assignee: Richard Ding
> Attachments: PIG-1075.patch
>
>
> When Cogrouping 2 relations on multiple key fields, pig throws an error if the corresponding types don't match.
> Consider the following script:-
> A = LOAD 'data' USING PigStorage() as (a:chararray, b:int, c:int);
> B = LOAD 'data' USING PigStorage() as (a:chararray, b:chararray, c:int);
> C = CoGROUP A BY (a,b,c), B BY (a,b,c);
> D = FOREACH C GENERATE FLATTEN(A), FLATTEN(B);
> describe D;
> dump D;
> The complete stack trace of the error thrown is
> Pig Stack Trace
> ---------------
> ERROR 1051: Cannot cast to Unknown
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1001: Unable to describe schema for alias D
> at org.apache.pig.PigServer.dumpSchema(PigServer.java:436)
> at org.apache.pig.tools.grunt.GruntParser.processDescribe(GruntParser.java:233)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:253)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:144)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89)
> at org.apache.pig.Main.main(Main.java:397)
> Caused by: org.apache.pig.impl.plan.PlanValidationException: ERROR 0: An unexpected exception caused the validation to stop
> at org.apache.pig.impl.plan.PlanValidator.validateSkipCollectException(PlanValidator.java:104)
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingValidator.validate(TypeCheckingValidator.java:40)
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingValidator.validate(TypeCheckingValidator.java:30)
> at org.apache.pig.impl.logicalLayer.validators.LogicalPlanValidationExecutor.validate(LogicalPlanValidationExecutor.java:83)
> at org.apache.pig.PigServer.compileLp(PigServer.java:821)
> at org.apache.pig.PigServer.dumpSchema(PigServer.java:428)
> ... 6 more
> Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: ERROR 1060: Cannot resolve COGroup output schema
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.visit(TypeCheckingVisitor.java:2463)
> at org.apache.pig.impl.logicalLayer.LOCogroup.visit(LOCogroup.java:372)
> at org.apache.pig.impl.logicalLayer.LOCogroup.visit(LOCogroup.java:45)
> at org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:69)
> at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
> at org.apache.pig.impl.plan.PlanValidator.validateSkipCollectException(PlanValidator.java:101)
> ... 11 more
> Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: ERROR 1051: Cannot cast to Unknown
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.insertAtomicCastForCOGroupInnerPlan(TypeCheckingVisitor.java:2552)
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.visit(TypeCheckingVisitor.java:2451)
> ... 16 more
> The error message does not help the user in identifying the issue clearly especially if the pig script is large and complex.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1075) Error in Cogroup when key fields types
don't match
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich updated PIG-1075:
--------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
patch committed. thanks Richard!
> Error in Cogroup when key fields types don't match
> --------------------------------------------------
>
> Key: PIG-1075
> URL: https://issues.apache.org/jira/browse/PIG-1075
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.5.0
> Reporter: Ankur
> Assignee: Richard Ding
> Fix For: 0.7.0
>
> Attachments: PIG-1075.patch
>
>
> When Cogrouping 2 relations on multiple key fields, pig throws an error if the corresponding types don't match.
> Consider the following script:-
> A = LOAD 'data' USING PigStorage() as (a:chararray, b:int, c:int);
> B = LOAD 'data' USING PigStorage() as (a:chararray, b:chararray, c:int);
> C = CoGROUP A BY (a,b,c), B BY (a,b,c);
> D = FOREACH C GENERATE FLATTEN(A), FLATTEN(B);
> describe D;
> dump D;
> The complete stack trace of the error thrown is
> Pig Stack Trace
> ---------------
> ERROR 1051: Cannot cast to Unknown
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1001: Unable to describe schema for alias D
> at org.apache.pig.PigServer.dumpSchema(PigServer.java:436)
> at org.apache.pig.tools.grunt.GruntParser.processDescribe(GruntParser.java:233)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:253)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:144)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89)
> at org.apache.pig.Main.main(Main.java:397)
> Caused by: org.apache.pig.impl.plan.PlanValidationException: ERROR 0: An unexpected exception caused the validation to stop
> at org.apache.pig.impl.plan.PlanValidator.validateSkipCollectException(PlanValidator.java:104)
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingValidator.validate(TypeCheckingValidator.java:40)
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingValidator.validate(TypeCheckingValidator.java:30)
> at org.apache.pig.impl.logicalLayer.validators.LogicalPlanValidationExecutor.validate(LogicalPlanValidationExecutor.java:83)
> at org.apache.pig.PigServer.compileLp(PigServer.java:821)
> at org.apache.pig.PigServer.dumpSchema(PigServer.java:428)
> ... 6 more
> Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: ERROR 1060: Cannot resolve COGroup output schema
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.visit(TypeCheckingVisitor.java:2463)
> at org.apache.pig.impl.logicalLayer.LOCogroup.visit(LOCogroup.java:372)
> at org.apache.pig.impl.logicalLayer.LOCogroup.visit(LOCogroup.java:45)
> at org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:69)
> at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
> at org.apache.pig.impl.plan.PlanValidator.validateSkipCollectException(PlanValidator.java:101)
> ... 11 more
> Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: ERROR 1051: Cannot cast to Unknown
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.insertAtomicCastForCOGroupInnerPlan(TypeCheckingVisitor.java:2552)
> at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.visit(TypeCheckingVisitor.java:2451)
> ... 16 more
> The error message does not help the user in identifying the issue clearly especially if the pig script is large and complex.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.