You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Christopher Olston (JIRA)" <ji...@apache.org> on 2009/03/24 21:17:50 UTC
[jira] Updated: (PIG-730) problem combining schema from a union of
several LOAD expressions, with a nested bag inside the schema.
[ https://issues.apache.org/jira/browse/PIG-730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Christopher Olston updated PIG-730:
-----------------------------------
Description:
grunt> a = load 'foo' using BinStorage as (url:chararray,outlinks:{t:(target:chararray,text:chararray)});
grunt> b = union (load 'foo' using BinStorage as (url:chararray,outlinks:{t:(target:chararray,text:chararray)})), (load 'bar' using BinStorage as (url:chararray,outlinks:{t:(target:chararray,text:chararray)}));
grunt> c = foreach a generate flatten(outlinks.target);
grunt> d = foreach b generate flatten(outlinks.target);
---> Would expect both C and D to work, but only C works. D gives the following error:
---> Turns out using outlinks.t.target works for D but not for C.
---> I don't care which one, but the same syntax should work for both!
2009-03-24 13:15:05,376 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1000: Error during parsing. Invalid alias: target in {t: (target: chararray,text: chararray)}
Details at logfile: /echo/olston/data/pig_1237925683748.log
grunt> quit
$ cat pig_1237925683748.log
ERROR 1000: Error during parsing. Invalid alias: target in {t: (target: chararray,text: chararray)}
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during parsing. Invalid alias: target in {t: (target: chararray,text: chararray)}
at org.apache.pig.PigServer.parseQuery(PigServer.java:317)
at org.apache.pig.PigServer.registerQuery(PigServer.java:276)
at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:529)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:280)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:99)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
at org.apache.pig.Main.main(Main.java:321)
Caused by: org.apache.pig.impl.logicalLayer.parser.ParseException: Invalid alias: target in {t: (target: chararray,text: chararray)}
at org.apache.pig.impl.logicalLayer.parser.QueryParser.AliasFieldOrSpec(QueryParser.java:6042)
at org.apache.pig.impl.logicalLayer.parser.QueryParser.ColOrSpec(QueryParser.java:5898)
at org.apache.pig.impl.logicalLayer.parser.QueryParser.BracketedSimpleProj(QueryParser.java:5423)
at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseEvalSpec(QueryParser.java:4100)
at org.apache.pig.impl.logicalLayer.parser.QueryParser.UnaryExpr(QueryParser.java:3967)
at org.apache.pig.impl.logicalLayer.parser.QueryParser.CastExpr(QueryParser.java:3920)
at org.apache.pig.impl.logicalLayer.parser.QueryParser.MultiplicativeExpr(QueryParser.java:3829)
at org.apache.pig.impl.logicalLayer.parser.QueryParser.AdditiveExpr(QueryParser.java:3755)
at org.apache.pig.impl.logicalLayer.parser.QueryParser.InfixExpr(QueryParser.java:3721)
at org.apache.pig.impl.logicalLayer.parser.QueryParser.FlattenedGenerateItem(QueryParser.java:3617)
at org.apache.pig.impl.logicalLayer.parser.QueryParser.FlattenedGenerateItemList(QueryParser.java:3557)
at org.apache.pig.impl.logicalLayer.parser.QueryParser.GenerateStatement(QueryParser.java:3514)
at org.apache.pig.impl.logicalLayer.parser.QueryParser.NestedBlock(QueryParser.java:2985)
at org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParser.java:2395)
at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1028)
at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:804)
at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:595)
at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:60)
at org.apache.pig.PigServer.parseQuery(PigServer.java:310)
... 6 more
was:
links = (union (load '/echo/olston/data/Echo-fetcher_first_extractedLinks-1' using BinStorage as (url:chararray,outlinks:{t:(target:chararray,text:chararray)})), (load '/echo/olston/data/Echo-fetcher_first_extractedLinks-2' using BinStorage as (url:chararray,outlinks:{t:(target:chararray,text:chararray)})));
DESCRIBE links;
gives this error:
2009-03-24 12:33:59,657 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1052: Cannot cast bag with schema outlinks: bag({t: (target: chararray,text: chararray)}) to tuple with schema tuple
Details at logfile: /echo/olston/data/pig_1237923219665.log
the pig_...log file contains:
ERROR 1052: Cannot cast bag with schema outlinks: bag({t: (target: chararray,text: chararray)}) to tuple with schema tuple
org.apache.pig.impl.logicalLayer.FrontendException: Unable to describe schema for alias links
at org.apache.pig.PigServer.dumpSchema(PigServer.java:394)
at org.apache.pig.tools.grunt.GruntParser.processDescribe(GruntParser.java:153)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:188)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:84)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:72)
at org.apache.pig.Main.main(Main.java:296)
Caused by: org.apache.pig.impl.plan.PlanValidationException: An unexpected exception caused the validation to stop
at org.apache.pig.impl.plan.PlanValidator.validateSkipCollectException(PlanValidator.java:104)
at org.apache.pig.impl.logicalLayer.validators.TypeCheckingValidator.validate(TypeCheckingValidator.java:40)
at org.apache.pig.impl.logicalLayer.validators.TypeCheckingValidator.validate(TypeCheckingValidator.java:30)
at org.apache.pig.impl.logicalLayer.validators.LogicalPlanValidationExecutor.validate(LogicalPlanValidationExecutor.java:79)
at org.apache.pig.PigServer.compileLp(PigServer.java:722)
at org.apache.pig.PigServer.dumpSchema(PigServer.java:386)
... 5 more
Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: Problem while casting inputs of Union
at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.visit(TypeCheckingVisitor.java:2143)
at org.apache.pig.impl.logicalLayer.LOUnion.visit(LOUnion.java:113)
at org.apache.pig.impl.logicalLayer.LOUnion.visit(LOUnion.java:34)
at org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:68)
at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
at org.apache.pig.impl.plan.PlanValidator.validateSkipCollectException(PlanValidator.java:101)
... 10 more
Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: Cannot cast bag with schema outlinks: bag({t: (target: chararray,text: chararray)}) to tuple with schema tuple
at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.visit(TypeCheckingVisitor.java:2034)
at org.apache.pig.impl.logicalLayer.LOCast.visit(LOCast.java:65)
at org.apache.pig.impl.logicalLayer.LOCast.visit(LOCast.java:29)
at org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:68)
at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.checkInnerPlan(TypeCheckingVisitor.java:2944)
at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.visit(TypeCheckingVisitor.java:2849)
at org.apache.pig.impl.logicalLayer.validators.TypeCheckingVisitor.visit(TypeCheckingVisitor.java:2137)
... 15 more
> problem combining schema from a union of several LOAD expressions, with a nested bag inside the schema.
> -------------------------------------------------------------------------------------------------------
>
> Key: PIG-730
> URL: https://issues.apache.org/jira/browse/PIG-730
> Project: Pig
> Issue Type: Bug
> Environment: pig local mode
> Reporter: Christopher Olston
>
> grunt> a = load 'foo' using BinStorage as (url:chararray,outlinks:{t:(target:chararray,text:chararray)});
> grunt> b = union (load 'foo' using BinStorage as (url:chararray,outlinks:{t:(target:chararray,text:chararray)})), (load 'bar' using BinStorage as (url:chararray,outlinks:{t:(target:chararray,text:chararray)}));
> grunt> c = foreach a generate flatten(outlinks.target);
> grunt> d = foreach b generate flatten(outlinks.target);
> ---> Would expect both C and D to work, but only C works. D gives the following error:
> ---> Turns out using outlinks.t.target works for D but not for C.
> ---> I don't care which one, but the same syntax should work for both!
> 2009-03-24 13:15:05,376 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1000: Error during parsing. Invalid alias: target in {t: (target: chararray,text: chararray)}
> Details at logfile: /echo/olston/data/pig_1237925683748.log
> grunt> quit
> $ cat pig_1237925683748.log
> ERROR 1000: Error during parsing. Invalid alias: target in {t: (target: chararray,text: chararray)}
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during parsing. Invalid alias: target in {t: (target: chararray,text: chararray)}
> at org.apache.pig.PigServer.parseQuery(PigServer.java:317)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:276)
> at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:529)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:280)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:99)
> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
> at org.apache.pig.Main.main(Main.java:321)
> Caused by: org.apache.pig.impl.logicalLayer.parser.ParseException: Invalid alias: target in {t: (target: chararray,text: chararray)}
> at org.apache.pig.impl.logicalLayer.parser.QueryParser.AliasFieldOrSpec(QueryParser.java:6042)
> at org.apache.pig.impl.logicalLayer.parser.QueryParser.ColOrSpec(QueryParser.java:5898)
> at org.apache.pig.impl.logicalLayer.parser.QueryParser.BracketedSimpleProj(QueryParser.java:5423)
> at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseEvalSpec(QueryParser.java:4100)
> at org.apache.pig.impl.logicalLayer.parser.QueryParser.UnaryExpr(QueryParser.java:3967)
> at org.apache.pig.impl.logicalLayer.parser.QueryParser.CastExpr(QueryParser.java:3920)
> at org.apache.pig.impl.logicalLayer.parser.QueryParser.MultiplicativeExpr(QueryParser.java:3829)
> at org.apache.pig.impl.logicalLayer.parser.QueryParser.AdditiveExpr(QueryParser.java:3755)
> at org.apache.pig.impl.logicalLayer.parser.QueryParser.InfixExpr(QueryParser.java:3721)
> at org.apache.pig.impl.logicalLayer.parser.QueryParser.FlattenedGenerateItem(QueryParser.java:3617)
> at org.apache.pig.impl.logicalLayer.parser.QueryParser.FlattenedGenerateItemList(QueryParser.java:3557)
> at org.apache.pig.impl.logicalLayer.parser.QueryParser.GenerateStatement(QueryParser.java:3514)
> at org.apache.pig.impl.logicalLayer.parser.QueryParser.NestedBlock(QueryParser.java:2985)
> at org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParser.java:2395)
> at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1028)
> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:804)
> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:595)
> at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:60)
> at org.apache.pig.PigServer.parseQuery(PigServer.java:310)
> ... 6 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.