[jira] [Updated] (PIG-5235) Typecast with as-clause fails for tuple/bag with an empty schema

["Koji Noguchi (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PIG-5235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Koji Noguchi updated PIG-5235:
------------------------------
    Attachment: pig-5235-v01.patch

Log file showed
{noformat}
Pig Stack Trace
---------------
ERROR 2999: Unexpected internal error. null

java.lang.NullPointerException
        at org.apache.pig.newplan.logical.relational.LogicalSchema$LogicalFieldSchema.typeMatch(LogicalSchema.java:174)
        at org.apache.pig.newplan.logical.visitor.ForEachUserSchemaVisitor.visit(ForEachUserSchemaVisitor.java:190)
        at org.apache.pig.newplan.logical.relational.LOForEach.accept(LOForEach.java:87)
        at org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75)
        at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:52)
        at org.apache.pig.newplan.logical.relational.LogicalPlan.validate(LogicalPlan.java:179)
        at org.apache.pig.PigServer$Graph.compile(PigServer.java:1851)
        at org.apache.pig.PigServer$Graph.access$300(PigServer.java:1527)
        at org.apache.pig.PigServer.execute(PigServer.java:1440)
        at org.apache.pig.PigServer.executeBatch(PigServer.java:488)
        at org.apache.pig.PigServer.executeBatch(PigServer.java:471)
        at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:170)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:233)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:204)
        at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
        at org.apache.pig.Main.run(Main.java:630)
        at org.apache.pig.Main.main(Main.java:175)
{noformat}

First issue is, {{LogicalSchema.typeMatch}} was not properly comparing the null schema.  Fixed.

After that, script failed with
{noformat}
Caused by: org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: ERROR 1052: Cannot cast tuple with schema mytuple2:tuple(name:NULL,age:NULL) to tuple with schema mytuple2:tuple(name:int,age:double)
        at org.apache.pig.newplan.logical.visitor.TypeCheckingExpVisitor.visit(TypeCheckingExpVisitor.java:518)
        at org.apache.pig.newplan.logical.expression.CastExpression.accept(CastExpression.java:44)
        at org.apache.pig.newplan.ReverseDependencyOrderWalker.walk(ReverseDependencyOrderWalker.java:70)
        at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:52)
        at org.apache.pig.newplan.logical.visitor.TypeCheckingRelVisitor.visitExpressionPlan(TypeCheckingRelVisitor.java:191)
        at org.apache.pig.newplan.logical.visitor.TypeCheckingRelVisitor.visit(TypeCheckingRelVisitor.java:157)
        at org.apache.pig.newplan.logical.relational.LOGenerate.accept(LOGenerate.java:254)
        at org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75)
        at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:52)
        at org.apache.pig.newplan.logical.visitor.TypeCheckingRelVisitor.visit(TypeCheckingRelVisitor.java:174)
        ... 15 more
{noformat}

In LOGenearte, we're converting NULL type to BYTEARRAY for simple types but not for nested-types with TUPLE and BAGs.  Fixed. 

Also while debugging, I got confused on why the schema didn't change after ForEachUserSchemaVisitor.  I added extra {{resetSchema}} for consistency purposes although schema was properly reset after this phase already.



> Typecast with as-clause fails for tuple/bag with an empty schema
> ----------------------------------------------------------------
>
>                 Key: PIG-5235
>                 URL: https://issues.apache.org/jira/browse/PIG-5235
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Koji Noguchi
>            Assignee: Koji Noguchi
>         Attachments: pig-5235-v01.patch
>
>
> Following script fails with trunk(0.17).
> {code}
> a = load 'test.txt' as (mytuple:tuple (), gpa:float);
> b = foreach a generate mytuple as (mytuple2:(name:int, age:double));
> store b into '/tmp/deleteme';
> {code}
> 2017-05-16 09:52:31,280 \[main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2999: Unexpected internal error. null
> (This is a continuation from the as-clause fix at PIG-2315 and follow up jira PIG-4933)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)