You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Alan Gates (JIRA)" <ji...@apache.org> on 2008/07/14 23:34:31 UTC
[jira] Created: (PIG-310) Some nested order by queries fail in
logical to physical translator
Some nested order by queries fail in logical to physical translator
-------------------------------------------------------------------
Key: PIG-310
URL: https://issues.apache.org/jira/browse/PIG-310
Project: Pig
Issue Type: Bug
Affects Versions: types_branch
Reporter: Alan Gates
Fix For: types_branch
The query:
{code}
a = load 'myfile';
b = group a by $0;
c = foreach b {c1 = order $1 by $1; generate flatten(c1); };
store c into 'outfile'
{code}
dies with the error message:
java.io.IOException: Unable to store for alias: c [null]
at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:232)
at org.apache.pig.PigServer.compilePp(PigServer.java:556)
at org.apache.pig.PigServer.execute(PigServer.java:482)
at org.apache.pig.PigServer.store(PigServer.java:324)
at org.apache.pig.PigServer.store(PigServer.java:310)
at org.apache.pig.tools.grunt.GruntParser.processStore(GruntParser.java:173)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:317)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:77)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:58)
at org.apache.pig.Main.main(Main.java:311)
Caused by: org.apache.pig.backend.executionengine.ExecException
... 10 more
Caused by: org.apache.pig.impl.plan.VisitorException
at org.apache.pig.impl.logicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:726)
at org.apache.pig.impl.logicalLayer.LOSort.visit(LOSort.java:141)
at org.apache.pig.impl.logicalLayer.LOSort.visit(LOSort.java:35)
at org.apache.pig.impl.plan.DependencyOrderWalkerWOSeenChk.walk(DependencyOrderWalkerWOSeenChk.java:68)
at org.apache.pig.impl.logicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:651)
at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:87)
at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:36)
at org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:68)
at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:229)
... 9 more
Caused by: org.apache.pig.impl.plan.PlanException: Attempt to connect operator Project[tuple][1] - gates-Sat Jul 12 17:57:09 PDT 2008-16 which is not in the pla
at org.apache.pig.impl.plan.OperatorPlan.checkInPlan(OperatorPlan.java:254)
at org.apache.pig.impl.plan.OperatorPlan.connect(OperatorPlan.java:140)
at org.apache.pig.impl.physicalLayer.plans.PhysicalPlan.connect(PhysicalPlan.java:77)
at org.apache.pig.impl.logicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:723)
... 18 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-310) Some nested order by queries fail in
logical to physical translator
Posted by "Santhosh Srinivasan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Santhosh Srinivasan updated PIG-310:
------------------------------------
Patch Info: [Patch Available]
> Some nested order by queries fail in logical to physical translator
> -------------------------------------------------------------------
>
> Key: PIG-310
> URL: https://issues.apache.org/jira/browse/PIG-310
> Project: Pig
> Issue Type: Bug
> Affects Versions: types_branch
> Reporter: Alan Gates
> Assignee: Santhosh Srinivasan
> Priority: Critical
> Fix For: types_branch
>
> Attachments: sort_with_nested_project.patch
>
>
> The query:
> {code}
> a = load 'myfile';
> b = group a by $0;
> c = foreach b {c1 = order $1 by $1; generate flatten(c1); };
> store c into 'outfile'
> {code}
> dies with the error message:
> java.io.IOException: Unable to store for alias: c [null]
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:232)
> at org.apache.pig.PigServer.compilePp(PigServer.java:556)
> at org.apache.pig.PigServer.execute(PigServer.java:482)
> at org.apache.pig.PigServer.store(PigServer.java:324)
> at org.apache.pig.PigServer.store(PigServer.java:310)
> at org.apache.pig.tools.grunt.GruntParser.processStore(GruntParser.java:173)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:317)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:77)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:58)
> at org.apache.pig.Main.main(Main.java:311)
> Caused by: org.apache.pig.backend.executionengine.ExecException
> ... 10 more
> Caused by: org.apache.pig.impl.plan.VisitorException
> at org.apache.pig.impl.logicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:726)
> at org.apache.pig.impl.logicalLayer.LOSort.visit(LOSort.java:141)
> at org.apache.pig.impl.logicalLayer.LOSort.visit(LOSort.java:35)
> at org.apache.pig.impl.plan.DependencyOrderWalkerWOSeenChk.walk(DependencyOrderWalkerWOSeenChk.java:68)
> at org.apache.pig.impl.logicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:651)
> at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:87)
> at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:36)
> at org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:68)
> at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:229)
> ... 9 more
> Caused by: org.apache.pig.impl.plan.PlanException: Attempt to connect operator Project[tuple][1] - gates-Sat Jul 12 17:57:09 PDT 2008-16 which is not in the pla
> at org.apache.pig.impl.plan.OperatorPlan.checkInPlan(OperatorPlan.java:254)
> at org.apache.pig.impl.plan.OperatorPlan.connect(OperatorPlan.java:140)
> at org.apache.pig.impl.physicalLayer.plans.PhysicalPlan.connect(PhysicalPlan.java:77)
> at org.apache.pig.impl.logicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:723)
> ... 18 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-310) Some nested order by queries fail in
logical to physical translator
Posted by "Santhosh Srinivasan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Santhosh Srinivasan updated PIG-310:
------------------------------------
Attachment: sort_with_nested_project.patch
The patch fixes the following issue:
1. In LOSort's nested plans, the projections will not be connected to their inputs. Earlier, due to the connection, the same operator was playing dual roles:
i. Sort's input
ii. The sort column's input
Unit test cases that still fail are:
[junit] Running org.apache.pig.test.TestEvalPipeline
[junit] Tests run: 8, Failures: 0, Errors: 1, Time elapsed: 179.904 sec
[junit] Test org.apache.pig.test.TestEvalPipeline FAILED
[junit] Running org.apache.pig.test.TestFilterOpNumeric
[junit] Tests run: 8, Failures: 0, Errors: 1, Time elapsed: 56.124 sec
[junit] Test org.apache.pig.test.TestFilterOpNumeric FAILED
[junit] Running org.apache.pig.test.TestBuiltin
[junit] Tests run: 23, Failures: 1, Errors: 1, Time elapsed: 14.8 sec
[junit] Test org.apache.pig.test.TestBuiltin FAILED
[junit] Running org.apache.pig.test.TestStoreOld
[junit] Tests run: 3, Failures: 0, Errors: 2, Time elapsed: 21.453 sec
[junit] Test org.apache.pig.test.TestStoreOld FAILED
> Some nested order by queries fail in logical to physical translator
> -------------------------------------------------------------------
>
> Key: PIG-310
> URL: https://issues.apache.org/jira/browse/PIG-310
> Project: Pig
> Issue Type: Bug
> Affects Versions: types_branch
> Reporter: Alan Gates
> Assignee: Santhosh Srinivasan
> Priority: Critical
> Fix For: types_branch
>
> Attachments: sort_with_nested_project.patch
>
>
> The query:
> {code}
> a = load 'myfile';
> b = group a by $0;
> c = foreach b {c1 = order $1 by $1; generate flatten(c1); };
> store c into 'outfile'
> {code}
> dies with the error message:
> java.io.IOException: Unable to store for alias: c [null]
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:232)
> at org.apache.pig.PigServer.compilePp(PigServer.java:556)
> at org.apache.pig.PigServer.execute(PigServer.java:482)
> at org.apache.pig.PigServer.store(PigServer.java:324)
> at org.apache.pig.PigServer.store(PigServer.java:310)
> at org.apache.pig.tools.grunt.GruntParser.processStore(GruntParser.java:173)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:317)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:77)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:58)
> at org.apache.pig.Main.main(Main.java:311)
> Caused by: org.apache.pig.backend.executionengine.ExecException
> ... 10 more
> Caused by: org.apache.pig.impl.plan.VisitorException
> at org.apache.pig.impl.logicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:726)
> at org.apache.pig.impl.logicalLayer.LOSort.visit(LOSort.java:141)
> at org.apache.pig.impl.logicalLayer.LOSort.visit(LOSort.java:35)
> at org.apache.pig.impl.plan.DependencyOrderWalkerWOSeenChk.walk(DependencyOrderWalkerWOSeenChk.java:68)
> at org.apache.pig.impl.logicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:651)
> at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:87)
> at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:36)
> at org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:68)
> at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:229)
> ... 9 more
> Caused by: org.apache.pig.impl.plan.PlanException: Attempt to connect operator Project[tuple][1] - gates-Sat Jul 12 17:57:09 PDT 2008-16 which is not in the pla
> at org.apache.pig.impl.plan.OperatorPlan.checkInPlan(OperatorPlan.java:254)
> at org.apache.pig.impl.plan.OperatorPlan.connect(OperatorPlan.java:140)
> at org.apache.pig.impl.physicalLayer.plans.PhysicalPlan.connect(PhysicalPlan.java:77)
> at org.apache.pig.impl.logicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:723)
> ... 18 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-310) Some nested order by queries fail in
logical to physical translator
Posted by "Santhosh Srinivasan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12616212#action_12616212 ]
Santhosh Srinivasan commented on PIG-310:
-----------------------------------------
The issue has been fixed. The logical, type checked and physical plans for the query will now look like:
{noformat}
ForEach Test-Plan-Builder-8 Schema: {bytearray} Type: bag
| |
| Project Test-Plan-Builder-6 Projections: [*] Overloaded: false FieldSchema: c1: tuple Type: tuple
| Input: SORT Test-Plan-Builder-5|
| |---SORT Test-Plan-Builder-5 Schema: null Type: bag
| | |
| | Project Test-Plan-Builder-4 Projections: [1] Overloaded: false FieldSchema: bytearray Type: bytearray
| | Input: Project Test-Plan-Builder-3 Projections: [1] Overloaded: fals
| |
| |---Project Test-Plan-Builder-3 Projections: [1] Overloaded: false FieldSchema: a: bag Type: bag
| Input: CoGroup Test-Plan-Builder-2
|
|---CoGroup Test-Plan-Builder-2 Schema: {group: bytearray,a: {null}} Type: Unknown
| |
| Project Test-Plan-Builder-1 Projections: [0] Overloaded: false FieldSchema: bytearray Type: bytearray
| Input: Load Test-Plan-Builder-
|
|---Load Test-Plan-Builder-0 Schema: null Type: bag
Printing the logical plan
ForEach Test-Plan-Builder-8 Schema: {bytearray} Type: bag
| |
| Project Test-Plan-Builder-6 Projections: [*] Overloaded: false FieldSchema: c1: bag Type: bag
| Input: SORT Test-Plan-Builder-5|
| |---SORT Test-Plan-Builder-5 Schema: null Type: bag
| | |
| | Project Test-Plan-Builder-4 Projections: [1] Overloaded: false FieldSchema: bytearray Type: bytearray
| | Input: Project Test-Plan-Builder-3 Projections: [1] Overloaded: tru
| |
| |---Project Test-Plan-Builder-3 Projections: [1] Overloaded: true FieldSchema: a: tuple Type: tuple
| Input: CoGroup Test-Plan-Builder-2
|
|---CoGroup Test-Plan-Builder-2 Schema: {group: bytearray,a: (null)} Type: bag
| |
| Project Test-Plan-Builder-1 Projections: [0] Overloaded: false FieldSchema: bytearray Type: bytearray
| Input: Load Test-Plan-Builder-
|
|---Load Test-Plan-Builder-0 Schema: null Type: bag
Printing the physical plan
New For Each(true)[bag] - Test-Plan-Builder-18
| |
| Project[bag][*] - Test-Plan-Builder-17
| |
| |---POSort[bag]() - Test-Plan-Builder-16
| | |
| | Project[bytearray][1] - Test-Plan-Builder-15
| |
| |---Project[tuple][1] - Test-Plan-Builder-14
|
|---Package[tuple]{bytearray} - Test-Plan-Builder-11
|
|---Global Rearrange[tuple] - Test-Plan-Builder-10
|
|---Local Rearrange[tuple]{bytearray} - Test-Plan-Builder-12
| |
| Project[bytearray][0] - Test-Plan-Builder-13
|
|---Load(myfile:org.apache.pig.builtin.PigStorage) - Test-Plan-Builder-9
{noformat}
> Some nested order by queries fail in logical to physical translator
> -------------------------------------------------------------------
>
> Key: PIG-310
> URL: https://issues.apache.org/jira/browse/PIG-310
> Project: Pig
> Issue Type: Bug
> Affects Versions: types_branch
> Reporter: Alan Gates
> Assignee: Santhosh Srinivasan
> Priority: Critical
> Fix For: types_branch
>
>
> The query:
> {code}
> a = load 'myfile';
> b = group a by $0;
> c = foreach b {c1 = order $1 by $1; generate flatten(c1); };
> store c into 'outfile'
> {code}
> dies with the error message:
> java.io.IOException: Unable to store for alias: c [null]
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:232)
> at org.apache.pig.PigServer.compilePp(PigServer.java:556)
> at org.apache.pig.PigServer.execute(PigServer.java:482)
> at org.apache.pig.PigServer.store(PigServer.java:324)
> at org.apache.pig.PigServer.store(PigServer.java:310)
> at org.apache.pig.tools.grunt.GruntParser.processStore(GruntParser.java:173)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:317)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:77)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:58)
> at org.apache.pig.Main.main(Main.java:311)
> Caused by: org.apache.pig.backend.executionengine.ExecException
> ... 10 more
> Caused by: org.apache.pig.impl.plan.VisitorException
> at org.apache.pig.impl.logicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:726)
> at org.apache.pig.impl.logicalLayer.LOSort.visit(LOSort.java:141)
> at org.apache.pig.impl.logicalLayer.LOSort.visit(LOSort.java:35)
> at org.apache.pig.impl.plan.DependencyOrderWalkerWOSeenChk.walk(DependencyOrderWalkerWOSeenChk.java:68)
> at org.apache.pig.impl.logicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:651)
> at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:87)
> at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:36)
> at org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:68)
> at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:229)
> ... 9 more
> Caused by: org.apache.pig.impl.plan.PlanException: Attempt to connect operator Project[tuple][1] - gates-Sat Jul 12 17:57:09 PDT 2008-16 which is not in the pla
> at org.apache.pig.impl.plan.OperatorPlan.checkInPlan(OperatorPlan.java:254)
> at org.apache.pig.impl.plan.OperatorPlan.connect(OperatorPlan.java:140)
> at org.apache.pig.impl.physicalLayer.plans.PhysicalPlan.connect(PhysicalPlan.java:77)
> at org.apache.pig.impl.logicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:723)
> ... 18 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-310) Some nested order by queries fail in
logical to physical translator
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12614791#action_12614791 ]
Alan Gates commented on PIG-310:
--------------------------------
It looks to me like the logical plan isn't being constructed quite correctly. It appears that one project is being connected in two different places in the plan.
Here is the logical plan for the above script:
{code}
Logical Plan:
ForEach gates-Fri Jul 18 09:28:13 PDT 2008-8 Schema: {bytearray} Type: bag
| |
| Project gates-Fri Jul 18 09:28:13 PDT 2008-6 Projections: [*] Overloaded: false FieldSchema: CA: bag Type: bag
| Input: SORT gates-Fri Jul 18 09:28:13 PDT 2008-5|
| |---SORT gates-Fri Jul 18 09:28:13 PDT 2008-5 Schema: null Type: bag
| | |
| | Project gates-Fri Jul 18 09:28:13 PDT 2008-4 Projections: [0] Overloaded: false FieldSchema: bytearray Type: bytearray
| | Input: Project gates-Fri Jul 18 09:28:13 PDT 2008-3 Projections: [1] Overloaded: true|
| | |---Project gates-Fri Jul 18 09:28:13 PDT 2008-3 Projections: [1] Overloaded: true FieldSchema: A: tuple Type: tuple
| | Input: CoGroup gates-Fri Jul 18 09:28:13 PDT 2008-2
| |
| |---Project gates-Fri Jul 18 09:28:13 PDT 2008-3 Projections: [1] Overloaded: true FieldSchema: A: tuple Type: tuple
| Input: CoGroup gates-Fri Jul 18 09:28:13 PDT 2008-2
|
|---CoGroup gates-Fri Jul 18 09:28:13 PDT 2008-2 Schema: {group: bytearray,A: (null)} Type: bag
| |
| Project gates-Fri Jul 18 09:28:13 PDT 2008-1 Projections: [0] Overloaded: false FieldSchema: bytearray Type: bytearray
| Input: Load gates-Fri Jul 18 09:28:13 PDT 2008-
|
|---Load gates-Fri Jul 18 09:28:13 PDT 2008-0 Schema: null Type: bag
{code}
Notice that the project named "Project gates-Fri Jul 18 09:28:13 PDT 2008-3" is connected both directly to the sort (that is, it's sorts input), and it is in the plan of sort. This does not look correct. And if you change the script from "order by $1" to "order by *", then the two projects are different, and the script works.
Assigning to Santhosh to take a look. If this isn't an issue please assign it back to me.
> Some nested order by queries fail in logical to physical translator
> -------------------------------------------------------------------
>
> Key: PIG-310
> URL: https://issues.apache.org/jira/browse/PIG-310
> Project: Pig
> Issue Type: Bug
> Affects Versions: types_branch
> Reporter: Alan Gates
> Fix For: types_branch
>
>
> The query:
> {code}
> a = load 'myfile';
> b = group a by $0;
> c = foreach b {c1 = order $1 by $1; generate flatten(c1); };
> store c into 'outfile'
> {code}
> dies with the error message:
> java.io.IOException: Unable to store for alias: c [null]
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:232)
> at org.apache.pig.PigServer.compilePp(PigServer.java:556)
> at org.apache.pig.PigServer.execute(PigServer.java:482)
> at org.apache.pig.PigServer.store(PigServer.java:324)
> at org.apache.pig.PigServer.store(PigServer.java:310)
> at org.apache.pig.tools.grunt.GruntParser.processStore(GruntParser.java:173)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:317)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:77)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:58)
> at org.apache.pig.Main.main(Main.java:311)
> Caused by: org.apache.pig.backend.executionengine.ExecException
> ... 10 more
> Caused by: org.apache.pig.impl.plan.VisitorException
> at org.apache.pig.impl.logicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:726)
> at org.apache.pig.impl.logicalLayer.LOSort.visit(LOSort.java:141)
> at org.apache.pig.impl.logicalLayer.LOSort.visit(LOSort.java:35)
> at org.apache.pig.impl.plan.DependencyOrderWalkerWOSeenChk.walk(DependencyOrderWalkerWOSeenChk.java:68)
> at org.apache.pig.impl.logicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:651)
> at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:87)
> at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:36)
> at org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:68)
> at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:229)
> ... 9 more
> Caused by: org.apache.pig.impl.plan.PlanException: Attempt to connect operator Project[tuple][1] - gates-Sat Jul 12 17:57:09 PDT 2008-16 which is not in the pla
> at org.apache.pig.impl.plan.OperatorPlan.checkInPlan(OperatorPlan.java:254)
> at org.apache.pig.impl.plan.OperatorPlan.connect(OperatorPlan.java:140)
> at org.apache.pig.impl.physicalLayer.plans.PhysicalPlan.connect(PhysicalPlan.java:77)
> at org.apache.pig.impl.logicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:723)
> ... 18 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-310) Some nested order by queries fail in
logical to physical translator
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Gates updated PIG-310:
---------------------------
Priority: Critical (was: Major)
> Some nested order by queries fail in logical to physical translator
> -------------------------------------------------------------------
>
> Key: PIG-310
> URL: https://issues.apache.org/jira/browse/PIG-310
> Project: Pig
> Issue Type: Bug
> Affects Versions: types_branch
> Reporter: Alan Gates
> Assignee: Santhosh Srinivasan
> Priority: Critical
> Fix For: types_branch
>
>
> The query:
> {code}
> a = load 'myfile';
> b = group a by $0;
> c = foreach b {c1 = order $1 by $1; generate flatten(c1); };
> store c into 'outfile'
> {code}
> dies with the error message:
> java.io.IOException: Unable to store for alias: c [null]
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:232)
> at org.apache.pig.PigServer.compilePp(PigServer.java:556)
> at org.apache.pig.PigServer.execute(PigServer.java:482)
> at org.apache.pig.PigServer.store(PigServer.java:324)
> at org.apache.pig.PigServer.store(PigServer.java:310)
> at org.apache.pig.tools.grunt.GruntParser.processStore(GruntParser.java:173)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:317)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:77)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:58)
> at org.apache.pig.Main.main(Main.java:311)
> Caused by: org.apache.pig.backend.executionengine.ExecException
> ... 10 more
> Caused by: org.apache.pig.impl.plan.VisitorException
> at org.apache.pig.impl.logicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:726)
> at org.apache.pig.impl.logicalLayer.LOSort.visit(LOSort.java:141)
> at org.apache.pig.impl.logicalLayer.LOSort.visit(LOSort.java:35)
> at org.apache.pig.impl.plan.DependencyOrderWalkerWOSeenChk.walk(DependencyOrderWalkerWOSeenChk.java:68)
> at org.apache.pig.impl.logicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:651)
> at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:87)
> at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:36)
> at org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:68)
> at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:229)
> ... 9 more
> Caused by: org.apache.pig.impl.plan.PlanException: Attempt to connect operator Project[tuple][1] - gates-Sat Jul 12 17:57:09 PDT 2008-16 which is not in the pla
> at org.apache.pig.impl.plan.OperatorPlan.checkInPlan(OperatorPlan.java:254)
> at org.apache.pig.impl.plan.OperatorPlan.connect(OperatorPlan.java:140)
> at org.apache.pig.impl.physicalLayer.plans.PhysicalPlan.connect(PhysicalPlan.java:77)
> at org.apache.pig.impl.logicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:723)
> ... 18 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (PIG-310) Some nested order by queries fail in
logical to physical translator
Posted by "Pi Song (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pi Song resolved PIG-310.
-------------------------
Resolution: Fixed
Committed
Thanks Santhosh.
> Some nested order by queries fail in logical to physical translator
> -------------------------------------------------------------------
>
> Key: PIG-310
> URL: https://issues.apache.org/jira/browse/PIG-310
> Project: Pig
> Issue Type: Bug
> Affects Versions: types_branch
> Reporter: Alan Gates
> Assignee: Santhosh Srinivasan
> Priority: Critical
> Fix For: types_branch
>
> Attachments: sort_with_nested_project.patch
>
>
> The query:
> {code}
> a = load 'myfile';
> b = group a by $0;
> c = foreach b {c1 = order $1 by $1; generate flatten(c1); };
> store c into 'outfile'
> {code}
> dies with the error message:
> java.io.IOException: Unable to store for alias: c [null]
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:232)
> at org.apache.pig.PigServer.compilePp(PigServer.java:556)
> at org.apache.pig.PigServer.execute(PigServer.java:482)
> at org.apache.pig.PigServer.store(PigServer.java:324)
> at org.apache.pig.PigServer.store(PigServer.java:310)
> at org.apache.pig.tools.grunt.GruntParser.processStore(GruntParser.java:173)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:317)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:77)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:58)
> at org.apache.pig.Main.main(Main.java:311)
> Caused by: org.apache.pig.backend.executionengine.ExecException
> ... 10 more
> Caused by: org.apache.pig.impl.plan.VisitorException
> at org.apache.pig.impl.logicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:726)
> at org.apache.pig.impl.logicalLayer.LOSort.visit(LOSort.java:141)
> at org.apache.pig.impl.logicalLayer.LOSort.visit(LOSort.java:35)
> at org.apache.pig.impl.plan.DependencyOrderWalkerWOSeenChk.walk(DependencyOrderWalkerWOSeenChk.java:68)
> at org.apache.pig.impl.logicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:651)
> at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:87)
> at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:36)
> at org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:68)
> at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:229)
> ... 9 more
> Caused by: org.apache.pig.impl.plan.PlanException: Attempt to connect operator Project[tuple][1] - gates-Sat Jul 12 17:57:09 PDT 2008-16 which is not in the pla
> at org.apache.pig.impl.plan.OperatorPlan.checkInPlan(OperatorPlan.java:254)
> at org.apache.pig.impl.plan.OperatorPlan.connect(OperatorPlan.java:140)
> at org.apache.pig.impl.physicalLayer.plans.PhysicalPlan.connect(PhysicalPlan.java:77)
> at org.apache.pig.impl.logicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:723)
> ... 18 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (PIG-310) Some nested order by queries fail in
logical to physical translator
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Gates reassigned PIG-310:
------------------------------
Assignee: Santhosh Srinivasan
> Some nested order by queries fail in logical to physical translator
> -------------------------------------------------------------------
>
> Key: PIG-310
> URL: https://issues.apache.org/jira/browse/PIG-310
> Project: Pig
> Issue Type: Bug
> Affects Versions: types_branch
> Reporter: Alan Gates
> Assignee: Santhosh Srinivasan
> Fix For: types_branch
>
>
> The query:
> {code}
> a = load 'myfile';
> b = group a by $0;
> c = foreach b {c1 = order $1 by $1; generate flatten(c1); };
> store c into 'outfile'
> {code}
> dies with the error message:
> java.io.IOException: Unable to store for alias: c [null]
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:232)
> at org.apache.pig.PigServer.compilePp(PigServer.java:556)
> at org.apache.pig.PigServer.execute(PigServer.java:482)
> at org.apache.pig.PigServer.store(PigServer.java:324)
> at org.apache.pig.PigServer.store(PigServer.java:310)
> at org.apache.pig.tools.grunt.GruntParser.processStore(GruntParser.java:173)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:317)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:77)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:58)
> at org.apache.pig.Main.main(Main.java:311)
> Caused by: org.apache.pig.backend.executionengine.ExecException
> ... 10 more
> Caused by: org.apache.pig.impl.plan.VisitorException
> at org.apache.pig.impl.logicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:726)
> at org.apache.pig.impl.logicalLayer.LOSort.visit(LOSort.java:141)
> at org.apache.pig.impl.logicalLayer.LOSort.visit(LOSort.java:35)
> at org.apache.pig.impl.plan.DependencyOrderWalkerWOSeenChk.walk(DependencyOrderWalkerWOSeenChk.java:68)
> at org.apache.pig.impl.logicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:651)
> at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:87)
> at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:36)
> at org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:68)
> at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:229)
> ... 9 more
> Caused by: org.apache.pig.impl.plan.PlanException: Attempt to connect operator Project[tuple][1] - gates-Sat Jul 12 17:57:09 PDT 2008-16 which is not in the pla
> at org.apache.pig.impl.plan.OperatorPlan.checkInPlan(OperatorPlan.java:254)
> at org.apache.pig.impl.plan.OperatorPlan.connect(OperatorPlan.java:140)
> at org.apache.pig.impl.physicalLayer.plans.PhysicalPlan.connect(PhysicalPlan.java:77)
> at org.apache.pig.impl.logicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:723)
> ... 18 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.