You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Olga Natkovich (JIRA)" <ji...@apache.org> on 2008/09/09 02:31:44 UTC

[jira] Created: (PIG-424) nested foreach with flatten and agg gives an error

nested foreach with flatten and agg gives an error
--------------------------------------------------

                 Key: PIG-424
                 URL: https://issues.apache.org/jira/browse/PIG-424
             Project: Pig
          Issue Type: Bug
    Affects Versions: types_branch
            Reporter: Olga Natkovich
             Fix For: types_branch


Script:

A = load 'data' as (name, age, gpa);
B = group A by name;
C = foreach B{
D = order A by gpa;
generate flatten(D), MAX(A.age);}
store C into 'ciemo_4';

Error:

08/09/08 17:25:48 ERROR pig.PigServer: Problem resolving LOForEach schema org.apache.pig.builtin.MAX does not work with inputs of type {age: bytearray}
08/09/08 17:25:48 ERROR pig.PigServer: Severe problem found during validation org.apache.pig.impl.plan.PlanValidationException: An unexpected exception caused the validation to stop
08/09/08 17:25:48 ERROR grunt.GruntParser: java.io.IOException: Unable to store for alias: 12 [Problem resolving LOForEach schema org.apache.pig.builtin.MAX does not work with inputs of type {age: bytearray}Severe problem found during validation org.apache.pig.impl.plan.PlanValidationException: An unexpected exception caused the validation to stop]
        at org.apache.pig.PigServer.compileLp(PigServer.java:606)
        at org.apache.pig.PigServer.compileLp(PigServer.java:547)
        at org.apache.pig.PigServer.execute(PigServer.java:533)
        at org.apache.pig.PigServer.registerQuery(PigServer.java:283)
        at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:432)
        at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:242)
        at org.apache.pig.tools.grunt.GruntParser.parseContOnError(GruntParser.java:93)
        at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:58)
        at org.apache.pig.Main.main(Main.java:282)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Assigned: (PIG-424) nested foreach with flatten and agg gives an error

Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/PIG-424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Olga Natkovich reassigned PIG-424:
----------------------------------

    Assignee: Santhosh Srinivasan

> nested foreach with flatten and agg gives an error
> --------------------------------------------------
>
>                 Key: PIG-424
>                 URL: https://issues.apache.org/jira/browse/PIG-424
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: types_branch
>            Reporter: Olga Natkovich
>            Assignee: Santhosh Srinivasan
>             Fix For: types_branch
>
>
> Script:
> A = load 'data' as (name, age, gpa);
> B = group A by name;
> C = foreach B{
> D = order A by gpa;
> generate flatten(D), MAX(A.age);}
> store C into 'ciemo_4';
> Error:
> 08/09/08 17:25:48 ERROR pig.PigServer: Problem resolving LOForEach schema org.apache.pig.builtin.MAX does not work with inputs of type {age: bytearray}
> 08/09/08 17:25:48 ERROR pig.PigServer: Severe problem found during validation org.apache.pig.impl.plan.PlanValidationException: An unexpected exception caused the validation to stop
> 08/09/08 17:25:48 ERROR grunt.GruntParser: java.io.IOException: Unable to store for alias: 12 [Problem resolving LOForEach schema org.apache.pig.builtin.MAX does not work with inputs of type {age: bytearray}Severe problem found during validation org.apache.pig.impl.plan.PlanValidationException: An unexpected exception caused the validation to stop]
>         at org.apache.pig.PigServer.compileLp(PigServer.java:606)
>         at org.apache.pig.PigServer.compileLp(PigServer.java:547)
>         at org.apache.pig.PigServer.execute(PigServer.java:533)
>         at org.apache.pig.PigServer.registerQuery(PigServer.java:283)
>         at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:432)
>         at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:242)
>         at org.apache.pig.tools.grunt.GruntParser.parseContOnError(GruntParser.java:93)
>         at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:58)
>         at org.apache.pig.Main.main(Main.java:282)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Resolved: (PIG-424) nested foreach with flatten and agg gives an error

Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/PIG-424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Olga Natkovich resolved PIG-424.
--------------------------------

    Resolution: Fixed

patch committed, thanks santhosh

> nested foreach with flatten and agg gives an error
> --------------------------------------------------
>
>                 Key: PIG-424
>                 URL: https://issues.apache.org/jira/browse/PIG-424
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: types_branch
>            Reporter: Olga Natkovich
>            Assignee: Santhosh Srinivasan
>             Fix For: types_branch
>
>         Attachments: PIG-424.patch, PIG-424_1.patch
>
>
> Script:
> A = load 'data' as (name, age, gpa);
> B = group A by name;
> C = foreach B{
> D = order A by gpa;
> generate flatten(D), MAX(A.age);}
> store C into 'ciemo_4';
> Error:
> 08/09/08 17:25:48 ERROR pig.PigServer: Problem resolving LOForEach schema org.apache.pig.builtin.MAX does not work with inputs of type {age: bytearray}
> 08/09/08 17:25:48 ERROR pig.PigServer: Severe problem found during validation org.apache.pig.impl.plan.PlanValidationException: An unexpected exception caused the validation to stop
> 08/09/08 17:25:48 ERROR grunt.GruntParser: java.io.IOException: Unable to store for alias: 12 [Problem resolving LOForEach schema org.apache.pig.builtin.MAX does not work with inputs of type {age: bytearray}Severe problem found during validation org.apache.pig.impl.plan.PlanValidationException: An unexpected exception caused the validation to stop]
>         at org.apache.pig.PigServer.compileLp(PigServer.java:606)
>         at org.apache.pig.PigServer.compileLp(PigServer.java:547)
>         at org.apache.pig.PigServer.execute(PigServer.java:533)
>         at org.apache.pig.PigServer.registerQuery(PigServer.java:283)
>         at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:432)
>         at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:242)
>         at org.apache.pig.tools.grunt.GruntParser.parseContOnError(GruntParser.java:93)
>         at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:58)
>         at org.apache.pig.Main.main(Main.java:282)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (PIG-424) nested foreach with flatten and agg gives an error

Posted by "Santhosh Srinivasan (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/PIG-424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Santhosh Srinivasan updated PIG-424:
------------------------------------

    Patch Info: [Patch Available]

> nested foreach with flatten and agg gives an error
> --------------------------------------------------
>
>                 Key: PIG-424
>                 URL: https://issues.apache.org/jira/browse/PIG-424
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: types_branch
>            Reporter: Olga Natkovich
>            Assignee: Santhosh Srinivasan
>             Fix For: types_branch
>
>         Attachments: PIG-424.patch
>
>
> Script:
> A = load 'data' as (name, age, gpa);
> B = group A by name;
> C = foreach B{
> D = order A by gpa;
> generate flatten(D), MAX(A.age);}
> store C into 'ciemo_4';
> Error:
> 08/09/08 17:25:48 ERROR pig.PigServer: Problem resolving LOForEach schema org.apache.pig.builtin.MAX does not work with inputs of type {age: bytearray}
> 08/09/08 17:25:48 ERROR pig.PigServer: Severe problem found during validation org.apache.pig.impl.plan.PlanValidationException: An unexpected exception caused the validation to stop
> 08/09/08 17:25:48 ERROR grunt.GruntParser: java.io.IOException: Unable to store for alias: 12 [Problem resolving LOForEach schema org.apache.pig.builtin.MAX does not work with inputs of type {age: bytearray}Severe problem found during validation org.apache.pig.impl.plan.PlanValidationException: An unexpected exception caused the validation to stop]
>         at org.apache.pig.PigServer.compileLp(PigServer.java:606)
>         at org.apache.pig.PigServer.compileLp(PigServer.java:547)
>         at org.apache.pig.PigServer.execute(PigServer.java:533)
>         at org.apache.pig.PigServer.registerQuery(PigServer.java:283)
>         at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:432)
>         at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:242)
>         at org.apache.pig.tools.grunt.GruntParser.parseContOnError(GruntParser.java:93)
>         at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:58)
>         at org.apache.pig.Main.main(Main.java:282)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (PIG-424) nested foreach with flatten and agg gives an error

Posted by "Santhosh Srinivasan (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/PIG-424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Santhosh Srinivasan updated PIG-424:
------------------------------------

    Attachment: PIG-424.patch

Attached patch (PIG-424.patch) addresses the following:

1. The schema computation in LOProject used references from its predecessor instead of making a copy. The manipulation of the field schemas altered the schema of the predecessor

2. Added copy constructors for FieldSchema and Schema

All unit test cases pass.

> nested foreach with flatten and agg gives an error
> --------------------------------------------------
>
>                 Key: PIG-424
>                 URL: https://issues.apache.org/jira/browse/PIG-424
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: types_branch
>            Reporter: Olga Natkovich
>            Assignee: Santhosh Srinivasan
>             Fix For: types_branch
>
>         Attachments: PIG-424.patch
>
>
> Script:
> A = load 'data' as (name, age, gpa);
> B = group A by name;
> C = foreach B{
> D = order A by gpa;
> generate flatten(D), MAX(A.age);}
> store C into 'ciemo_4';
> Error:
> 08/09/08 17:25:48 ERROR pig.PigServer: Problem resolving LOForEach schema org.apache.pig.builtin.MAX does not work with inputs of type {age: bytearray}
> 08/09/08 17:25:48 ERROR pig.PigServer: Severe problem found during validation org.apache.pig.impl.plan.PlanValidationException: An unexpected exception caused the validation to stop
> 08/09/08 17:25:48 ERROR grunt.GruntParser: java.io.IOException: Unable to store for alias: 12 [Problem resolving LOForEach schema org.apache.pig.builtin.MAX does not work with inputs of type {age: bytearray}Severe problem found during validation org.apache.pig.impl.plan.PlanValidationException: An unexpected exception caused the validation to stop]
>         at org.apache.pig.PigServer.compileLp(PigServer.java:606)
>         at org.apache.pig.PigServer.compileLp(PigServer.java:547)
>         at org.apache.pig.PigServer.execute(PigServer.java:533)
>         at org.apache.pig.PigServer.registerQuery(PigServer.java:283)
>         at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:432)
>         at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:242)
>         at org.apache.pig.tools.grunt.GruntParser.parseContOnError(GruntParser.java:93)
>         at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:58)
>         at org.apache.pig.Main.main(Main.java:282)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (PIG-424) nested foreach with flatten and agg gives an error

Posted by "Santhosh Srinivasan (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/PIG-424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Santhosh Srinivasan updated PIG-424:
------------------------------------

    Attachment: PIG-424_1.patch

Modified the earlier patch to be consistent with the use of the copy constructor and made the patch current with SVN

Unit tests that still fail are:

    [junit] Running org.apache.pig.test.TestEvalPipeline
    [junit] Tests run: 8, Failures: 1, Errors: 0, Time elapsed: 176.18 sec
    [junit] Test org.apache.pig.test.TestEvalPipeline FAILED


> nested foreach with flatten and agg gives an error
> --------------------------------------------------
>
>                 Key: PIG-424
>                 URL: https://issues.apache.org/jira/browse/PIG-424
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: types_branch
>            Reporter: Olga Natkovich
>            Assignee: Santhosh Srinivasan
>             Fix For: types_branch
>
>         Attachments: PIG-424.patch, PIG-424_1.patch
>
>
> Script:
> A = load 'data' as (name, age, gpa);
> B = group A by name;
> C = foreach B{
> D = order A by gpa;
> generate flatten(D), MAX(A.age);}
> store C into 'ciemo_4';
> Error:
> 08/09/08 17:25:48 ERROR pig.PigServer: Problem resolving LOForEach schema org.apache.pig.builtin.MAX does not work with inputs of type {age: bytearray}
> 08/09/08 17:25:48 ERROR pig.PigServer: Severe problem found during validation org.apache.pig.impl.plan.PlanValidationException: An unexpected exception caused the validation to stop
> 08/09/08 17:25:48 ERROR grunt.GruntParser: java.io.IOException: Unable to store for alias: 12 [Problem resolving LOForEach schema org.apache.pig.builtin.MAX does not work with inputs of type {age: bytearray}Severe problem found during validation org.apache.pig.impl.plan.PlanValidationException: An unexpected exception caused the validation to stop]
>         at org.apache.pig.PigServer.compileLp(PigServer.java:606)
>         at org.apache.pig.PigServer.compileLp(PigServer.java:547)
>         at org.apache.pig.PigServer.execute(PigServer.java:533)
>         at org.apache.pig.PigServer.registerQuery(PigServer.java:283)
>         at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:432)
>         at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:242)
>         at org.apache.pig.tools.grunt.GruntParser.parseContOnError(GruntParser.java:93)
>         at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:58)
>         at org.apache.pig.Main.main(Main.java:282)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.