You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Viraj Bhat (JIRA)" <ji...@apache.org> on 2009/05/20 03:45:45 UTC
[jira] Created: (PIG-812) COUNT(*) does not work
COUNT(*) does not work
-----------------------
Key: PIG-812
URL: https://issues.apache.org/jira/browse/PIG-812
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.2.0
Reporter: Viraj Bhat
Priority: Critical
Fix For: 0.2.0
Pig script to count the number of rows in a studenttab10k file which contains 10k records.
{code}
studenttab = LOAD 'studenttab10k' AS (name:chararray, age:int,gpa:float);
X2 = GROUP studenttab ALL;
describe X2;
Y2 = FOREACH X2 GENERATE COUNT(*);
explain Y2;
DUMP Y2;
{code}
returns the following error
================================================================
ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias Y2
Details at logfile: /homes/viraj/pig-svn/trunk/pig_1242783700970.log
================================================================
If you look at the log file:
================================================================
Caused by: java.lang.ClassCastException
at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:76)
at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:68)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:201)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:235)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:254)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:223)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:245)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:236)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:88)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
================================================================
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-812) COUNT(*) does not work
Posted by "George Mavromatis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12711786#action_12711786 ]
George Mavromatis commented on PIG-812:
---------------------------------------
Olga's clarification makes sens but it is missing from the pig latin spec. Even Viraj did not know it. How can users know it?
In order for this to be resolved in the right manner the following must added in the http://hadoop.apache.org/pig/docs/r0.2.0/piglatin.html
1) The semantics of * as explained by Olga.
2) An example of GROUP ALL
Otherwise people will waste their time doing the same (documentation-caused) mistakes again.
> COUNT(*) does not work
> -----------------------
>
> Key: PIG-812
> URL: https://issues.apache.org/jira/browse/PIG-812
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.2.0
> Reporter: Viraj Bhat
> Priority: Critical
> Fix For: 0.2.0
>
> Attachments: studenttab10k
>
>
> Pig script to count the number of rows in a studenttab10k file which contains 10k records.
> {code}
> studenttab = LOAD 'studenttab10k' AS (name:chararray, age:int,gpa:float);
> X2 = GROUP studenttab ALL;
> describe X2;
> Y2 = FOREACH X2 GENERATE COUNT(*);
> explain Y2;
> DUMP Y2;
> {code}
> returns the following error
> ================================================================
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias Y2
> Details at logfile: /homes/viraj/pig-svn/trunk/pig_1242783700970.log
> ================================================================
> If you look at the log file:
> ================================================================
> Caused by: java.lang.ClassCastException
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:76)
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:68)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:201)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:235)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:254)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:223)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:245)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:236)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:88)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> ================================================================
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-812) COUNT(*) does not work
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12729550#action_12729550 ]
Hadoop QA commented on PIG-812:
-------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12413078/PIG-812.patch
against trunk revision 792663.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-minerva.apache.org/121/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-minerva.apache.org/121/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-minerva.apache.org/121/console
This message is automatically generated.
> COUNT(*) does not work
> -----------------------
>
> Key: PIG-812
> URL: https://issues.apache.org/jira/browse/PIG-812
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.2.0
> Reporter: Viraj Bhat
> Assignee: Benjamin Reed
> Fix For: 0.2.0
>
> Attachments: PIG-812.patch, PIG-812.pdf, studenttab10k
>
>
> Pig script to count the number of rows in a studenttab10k file which contains 10k records.
> {code}
> studenttab = LOAD 'studenttab10k' AS (name:chararray, age:int,gpa:float);
> X2 = GROUP studenttab ALL;
> describe X2;
> Y2 = FOREACH X2 GENERATE COUNT(*);
> explain Y2;
> DUMP Y2;
> {code}
> returns the following error
> ================================================================
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias Y2
> Details at logfile: /homes/viraj/pig-svn/trunk/pig_1242783700970.log
> ================================================================
> If you look at the log file:
> ================================================================
> Caused by: java.lang.ClassCastException
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:76)
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:68)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:201)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:235)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:254)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:223)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:245)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:236)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:88)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> ================================================================
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-812) COUNT(*) does not work
Posted by "Benjamin Reed (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benjamin Reed updated PIG-812:
------------------------------
Attachment: (was: PIG-812.pdf)
> COUNT(*) does not work
> -----------------------
>
> Key: PIG-812
> URL: https://issues.apache.org/jira/browse/PIG-812
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.2.0
> Reporter: Viraj Bhat
> Assignee: Benjamin Reed
> Fix For: 0.2.0
>
> Attachments: PIG-812.patch, PIG-812.pdf, studenttab10k
>
>
> Pig script to count the number of rows in a studenttab10k file which contains 10k records.
> {code}
> studenttab = LOAD 'studenttab10k' AS (name:chararray, age:int,gpa:float);
> X2 = GROUP studenttab ALL;
> describe X2;
> Y2 = FOREACH X2 GENERATE COUNT(*);
> explain Y2;
> DUMP Y2;
> {code}
> returns the following error
> ================================================================
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias Y2
> Details at logfile: /homes/viraj/pig-svn/trunk/pig_1242783700970.log
> ================================================================
> If you look at the log file:
> ================================================================
> Caused by: java.lang.ClassCastException
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:76)
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:68)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:201)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:235)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:254)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:223)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:245)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:236)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:88)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> ================================================================
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-812) COUNT(*) does not work
Posted by "Viraj Bhat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12710972#action_12710972 ]
Viraj Bhat commented on PIG-812:
--------------------------------
The workaround is to use:
{code}
Y2 = FOREACH X2 GENERATE COUNT(studenttab);
{code}
Please update the documentation at http://hadoop.apache.org/pig/docs/r0.2.0/piglatin.html if COUNT(*) is actually supported in the latest release. This is definitely a regression from the previous releases.
Thanks
Viraj
> COUNT(*) does not work
> -----------------------
>
> Key: PIG-812
> URL: https://issues.apache.org/jira/browse/PIG-812
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.2.0
> Reporter: Viraj Bhat
> Priority: Critical
> Fix For: 0.2.0
>
> Attachments: studenttab10k
>
>
> Pig script to count the number of rows in a studenttab10k file which contains 10k records.
> {code}
> studenttab = LOAD 'studenttab10k' AS (name:chararray, age:int,gpa:float);
> X2 = GROUP studenttab ALL;
> describe X2;
> Y2 = FOREACH X2 GENERATE COUNT(*);
> explain Y2;
> DUMP Y2;
> {code}
> returns the following error
> ================================================================
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias Y2
> Details at logfile: /homes/viraj/pig-svn/trunk/pig_1242783700970.log
> ================================================================
> If you look at the log file:
> ================================================================
> Caused by: java.lang.ClassCastException
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:76)
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:68)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:201)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:235)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:254)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:223)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:245)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:236)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:88)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> ================================================================
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-812) COUNT(*) does not work
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12745584#action_12745584 ]
Olga Natkovich commented on PIG-812:
------------------------------------
+1. There seems to be a consensus that this patch is an improvement. Ben, please, commit. We can make incremental improvements later
> COUNT(*) does not work
> -----------------------
>
> Key: PIG-812
> URL: https://issues.apache.org/jira/browse/PIG-812
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.2.0
> Reporter: Viraj Bhat
> Assignee: Benjamin Reed
> Fix For: 0.2.0
>
> Attachments: PIG-812.patch, PIG-812.patch, PIG-812.pdf, studenttab10k
>
>
> Pig script to count the number of rows in a studenttab10k file which contains 10k records.
> {code}
> studenttab = LOAD 'studenttab10k' AS (name:chararray, age:int,gpa:float);
> X2 = GROUP studenttab ALL;
> describe X2;
> Y2 = FOREACH X2 GENERATE COUNT(*);
> explain Y2;
> DUMP Y2;
> {code}
> returns the following error
> ================================================================
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias Y2
> Details at logfile: /homes/viraj/pig-svn/trunk/pig_1242783700970.log
> ================================================================
> If you look at the log file:
> ================================================================
> Caused by: java.lang.ClassCastException
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:76)
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:68)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:201)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:235)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:254)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:223)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:245)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:236)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:88)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> ================================================================
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-812) COUNT(*) does not work
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12746706#action_12746706 ]
Hudson commented on PIG-812:
----------------------------
Integrated in Pig-trunk #530 (See [http://hudson.zones.apache.org/hudson/job/Pig-trunk/530/])
. COUNT(*) does not work
> COUNT(*) does not work
> -----------------------
>
> Key: PIG-812
> URL: https://issues.apache.org/jira/browse/PIG-812
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.2.0
> Reporter: Viraj Bhat
> Assignee: Benjamin Reed
> Fix For: 0.2.0
>
> Attachments: PIG-812.patch, PIG-812.patch, PIG-812.pdf, studenttab10k
>
>
> Pig script to count the number of rows in a studenttab10k file which contains 10k records.
> {code}
> studenttab = LOAD 'studenttab10k' AS (name:chararray, age:int,gpa:float);
> X2 = GROUP studenttab ALL;
> describe X2;
> Y2 = FOREACH X2 GENERATE COUNT(*);
> explain Y2;
> DUMP Y2;
> {code}
> returns the following error
> ================================================================
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias Y2
> Details at logfile: /homes/viraj/pig-svn/trunk/pig_1242783700970.log
> ================================================================
> If you look at the log file:
> ================================================================
> Caused by: java.lang.ClassCastException
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:76)
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:68)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:201)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:235)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:254)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:223)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:245)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:236)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:88)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> ================================================================
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-812) COUNT(*) does not work
Posted by "Benjamin Reed (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benjamin Reed updated PIG-812:
------------------------------
Status: Patch Available (was: Reopened)
> COUNT(*) does not work
> -----------------------
>
> Key: PIG-812
> URL: https://issues.apache.org/jira/browse/PIG-812
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.2.0
> Reporter: Viraj Bhat
> Assignee: Benjamin Reed
> Fix For: 0.2.0
>
> Attachments: PIG-812.patch, PIG-812.pdf, studenttab10k
>
>
> Pig script to count the number of rows in a studenttab10k file which contains 10k records.
> {code}
> studenttab = LOAD 'studenttab10k' AS (name:chararray, age:int,gpa:float);
> X2 = GROUP studenttab ALL;
> describe X2;
> Y2 = FOREACH X2 GENERATE COUNT(*);
> explain Y2;
> DUMP Y2;
> {code}
> returns the following error
> ================================================================
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias Y2
> Details at logfile: /homes/viraj/pig-svn/trunk/pig_1242783700970.log
> ================================================================
> If you look at the log file:
> ================================================================
> Caused by: java.lang.ClassCastException
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:76)
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:68)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:201)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:235)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:254)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:223)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:245)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:236)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:88)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> ================================================================
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (PIG-812) COUNT(*) does not work
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich resolved PIG-812.
--------------------------------
Resolution: Won't Fix
The fact that this worked in earlier code was a bug. Now pig has a consistent implementation of *.
>From the beginning, Pig chose a different semantics for * than SQL. (it is unfortunate that we chose to use "*" for this but is something we need to leave in for consistency and backward compatibility.)
"*" in SQL means a relation while in pig it means a tuple passed to an operator. So in Pig you can order on the entire row by saying
B = order A by *;
You can also pass the entire row to a UDF by doing myUDF(*).
In this context COUNT(*) makes no sense.
> COUNT(*) does not work
> -----------------------
>
> Key: PIG-812
> URL: https://issues.apache.org/jira/browse/PIG-812
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.2.0
> Reporter: Viraj Bhat
> Priority: Critical
> Fix For: 0.2.0
>
> Attachments: studenttab10k
>
>
> Pig script to count the number of rows in a studenttab10k file which contains 10k records.
> {code}
> studenttab = LOAD 'studenttab10k' AS (name:chararray, age:int,gpa:float);
> X2 = GROUP studenttab ALL;
> describe X2;
> Y2 = FOREACH X2 GENERATE COUNT(*);
> explain Y2;
> DUMP Y2;
> {code}
> returns the following error
> ================================================================
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias Y2
> Details at logfile: /homes/viraj/pig-svn/trunk/pig_1242783700970.log
> ================================================================
> If you look at the log file:
> ================================================================
> Caused by: java.lang.ClassCastException
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:76)
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:68)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:201)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:235)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:254)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:223)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:245)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:236)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:88)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> ================================================================
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-812) COUNT(*) does not work
Posted by "Benjamin Reed (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benjamin Reed updated PIG-812:
------------------------------
Attachment: PIG-812.patch
small changes suggested by Corinne.
> COUNT(*) does not work
> -----------------------
>
> Key: PIG-812
> URL: https://issues.apache.org/jira/browse/PIG-812
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.2.0
> Reporter: Viraj Bhat
> Assignee: Benjamin Reed
> Fix For: 0.2.0
>
> Attachments: PIG-812.patch, PIG-812.patch, PIG-812.pdf, studenttab10k
>
>
> Pig script to count the number of rows in a studenttab10k file which contains 10k records.
> {code}
> studenttab = LOAD 'studenttab10k' AS (name:chararray, age:int,gpa:float);
> X2 = GROUP studenttab ALL;
> describe X2;
> Y2 = FOREACH X2 GENERATE COUNT(*);
> explain Y2;
> DUMP Y2;
> {code}
> returns the following error
> ================================================================
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias Y2
> Details at logfile: /homes/viraj/pig-svn/trunk/pig_1242783700970.log
> ================================================================
> If you look at the log file:
> ================================================================
> Caused by: java.lang.ClassCastException
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:76)
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:68)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:201)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:235)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:254)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:223)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:245)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:236)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:88)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> ================================================================
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-812) COUNT(*) does not work
Posted by "Benjamin Reed (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benjamin Reed updated PIG-812:
------------------------------
Attachment: PIG-812.pdf
PIG-812.patch
i've modified the doc to better explain the expressions that can be used in the functions and how flatten works. for convenience for reviewing i've attached a pdf of the changed pages.
> COUNT(*) does not work
> -----------------------
>
> Key: PIG-812
> URL: https://issues.apache.org/jira/browse/PIG-812
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.2.0
> Reporter: Viraj Bhat
> Assignee: Benjamin Reed
> Fix For: 0.2.0
>
> Attachments: PIG-812.patch, PIG-812.pdf, studenttab10k
>
>
> Pig script to count the number of rows in a studenttab10k file which contains 10k records.
> {code}
> studenttab = LOAD 'studenttab10k' AS (name:chararray, age:int,gpa:float);
> X2 = GROUP studenttab ALL;
> describe X2;
> Y2 = FOREACH X2 GENERATE COUNT(*);
> explain Y2;
> DUMP Y2;
> {code}
> returns the following error
> ================================================================
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias Y2
> Details at logfile: /homes/viraj/pig-svn/trunk/pig_1242783700970.log
> ================================================================
> If you look at the log file:
> ================================================================
> Caused by: java.lang.ClassCastException
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:76)
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:68)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:201)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:235)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:254)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:223)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:245)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:236)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:88)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> ================================================================
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-812) COUNT(*) does not work
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12734694#action_12734694 ]
Olga Natkovich commented on PIG-812:
------------------------------------
Ben, thanks for updating the docs. A couple of comments/suggestions:
(1) In Star expression section, I think it would be helpful to explain the difference between * in Pig and SQL in more details.
(2) Boolean, tuple, field, and general expression sections seems a little brief and I am not sure they add much to the user's understanding of the language. Perhaps examples would be helpful?
(3) Description of map dereferencing has key while the Symbol column says 'key'. I think that's confusing.
(4) The flatten description for a bag is not very clear and I also think has a typo: ({(b,c),(d,e)}) - I think the parenthesis are wrong - I think you meant to have a bag with a tuple that contains other tuples, right?
(5) Group vs. Cogroup - I think we should put all the information under COUGROUP because we always sold that as the general case and GROUP as "alias" for 1 relation case.
> COUNT(*) does not work
> -----------------------
>
> Key: PIG-812
> URL: https://issues.apache.org/jira/browse/PIG-812
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.2.0
> Reporter: Viraj Bhat
> Assignee: Benjamin Reed
> Fix For: 0.2.0
>
> Attachments: PIG-812.patch, PIG-812.pdf, studenttab10k
>
>
> Pig script to count the number of rows in a studenttab10k file which contains 10k records.
> {code}
> studenttab = LOAD 'studenttab10k' AS (name:chararray, age:int,gpa:float);
> X2 = GROUP studenttab ALL;
> describe X2;
> Y2 = FOREACH X2 GENERATE COUNT(*);
> explain Y2;
> DUMP Y2;
> {code}
> returns the following error
> ================================================================
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias Y2
> Details at logfile: /homes/viraj/pig-svn/trunk/pig_1242783700970.log
> ================================================================
> If you look at the log file:
> ================================================================
> Caused by: java.lang.ClassCastException
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:76)
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:68)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:201)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:235)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:254)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:223)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:245)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:236)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:88)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> ================================================================
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-812) COUNT(*) does not work
Posted by "Benjamin Reed (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benjamin Reed updated PIG-812:
------------------------------
Attachment: PIG-812.pdf
> COUNT(*) does not work
> -----------------------
>
> Key: PIG-812
> URL: https://issues.apache.org/jira/browse/PIG-812
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.2.0
> Reporter: Viraj Bhat
> Assignee: Benjamin Reed
> Fix For: 0.2.0
>
> Attachments: PIG-812.patch, PIG-812.pdf, studenttab10k
>
>
> Pig script to count the number of rows in a studenttab10k file which contains 10k records.
> {code}
> studenttab = LOAD 'studenttab10k' AS (name:chararray, age:int,gpa:float);
> X2 = GROUP studenttab ALL;
> describe X2;
> Y2 = FOREACH X2 GENERATE COUNT(*);
> explain Y2;
> DUMP Y2;
> {code}
> returns the following error
> ================================================================
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias Y2
> Details at logfile: /homes/viraj/pig-svn/trunk/pig_1242783700970.log
> ================================================================
> If you look at the log file:
> ================================================================
> Caused by: java.lang.ClassCastException
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:76)
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:68)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:201)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:235)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:254)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:223)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:245)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:236)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:88)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> ================================================================
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-812) COUNT(*) does not work
Posted by "Benjamin Reed (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benjamin Reed updated PIG-812:
------------------------------
Attachment: PIG-812.patch
Reattaching the patch so that it gets applied rather than the PDF.
> COUNT(*) does not work
> -----------------------
>
> Key: PIG-812
> URL: https://issues.apache.org/jira/browse/PIG-812
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.2.0
> Reporter: Viraj Bhat
> Assignee: Benjamin Reed
> Fix For: 0.2.0
>
> Attachments: PIG-812.patch, PIG-812.pdf, studenttab10k
>
>
> Pig script to count the number of rows in a studenttab10k file which contains 10k records.
> {code}
> studenttab = LOAD 'studenttab10k' AS (name:chararray, age:int,gpa:float);
> X2 = GROUP studenttab ALL;
> describe X2;
> Y2 = FOREACH X2 GENERATE COUNT(*);
> explain Y2;
> DUMP Y2;
> {code}
> returns the following error
> ================================================================
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias Y2
> Details at logfile: /homes/viraj/pig-svn/trunk/pig_1242783700970.log
> ================================================================
> If you look at the log file:
> ================================================================
> Caused by: java.lang.ClassCastException
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:76)
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:68)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:201)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:235)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:254)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:223)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:245)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:236)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:88)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> ================================================================
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Reopened: (PIG-812) COUNT(*) does not work
Posted by "Benjamin Reed (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benjamin Reed reopened PIG-812:
-------------------------------
Assignee: Benjamin Reed
updating the docs to reflect this semantics.
> COUNT(*) does not work
> -----------------------
>
> Key: PIG-812
> URL: https://issues.apache.org/jira/browse/PIG-812
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.2.0
> Reporter: Viraj Bhat
> Assignee: Benjamin Reed
> Priority: Critical
> Fix For: 0.2.0
>
> Attachments: studenttab10k
>
>
> Pig script to count the number of rows in a studenttab10k file which contains 10k records.
> {code}
> studenttab = LOAD 'studenttab10k' AS (name:chararray, age:int,gpa:float);
> X2 = GROUP studenttab ALL;
> describe X2;
> Y2 = FOREACH X2 GENERATE COUNT(*);
> explain Y2;
> DUMP Y2;
> {code}
> returns the following error
> ================================================================
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias Y2
> Details at logfile: /homes/viraj/pig-svn/trunk/pig_1242783700970.log
> ================================================================
> If you look at the log file:
> ================================================================
> Caused by: java.lang.ClassCastException
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:76)
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:68)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:201)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:235)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:254)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:223)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:245)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:236)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:88)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> ================================================================
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-812) COUNT(*) does not work
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12729115#action_12729115 ]
Hadoop QA commented on PIG-812:
-------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12412966/PIG-812.pdf
against trunk revision 792368.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
-1 patch. The patch command could not apply the patch.
Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-minerva.apache.org/118/console
This message is automatically generated.
> COUNT(*) does not work
> -----------------------
>
> Key: PIG-812
> URL: https://issues.apache.org/jira/browse/PIG-812
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.2.0
> Reporter: Viraj Bhat
> Assignee: Benjamin Reed
> Fix For: 0.2.0
>
> Attachments: PIG-812.patch, PIG-812.pdf, studenttab10k
>
>
> Pig script to count the number of rows in a studenttab10k file which contains 10k records.
> {code}
> studenttab = LOAD 'studenttab10k' AS (name:chararray, age:int,gpa:float);
> X2 = GROUP studenttab ALL;
> describe X2;
> Y2 = FOREACH X2 GENERATE COUNT(*);
> explain Y2;
> DUMP Y2;
> {code}
> returns the following error
> ================================================================
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias Y2
> Details at logfile: /homes/viraj/pig-svn/trunk/pig_1242783700970.log
> ================================================================
> If you look at the log file:
> ================================================================
> Caused by: java.lang.ClassCastException
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:76)
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:68)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:201)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:235)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:254)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:223)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:245)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:236)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:88)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> ================================================================
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-812) COUNT(*) does not work
Posted by "George Mavromatis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
George Mavromatis updated PIG-812:
----------------------------------
First, I have to say that the updated documentation specifies adequately further undocumented but important details of pig syntax and behavior (e.g. dereference) in addition to the ones mentioned in this ticket. Very good job! This will improve usability of pig significantly. Thank you.
Regarding the particular points in this ticket:
1) Description is adequate for pig. Contrast with sql is nice to have, not crucial.
2) These parts were probably edited by Corinne. Examples may help, but again the spec is important to have and that was addressed.
3) 'key' or key should be consistent in both "symbol" and "notes" sections.
4) I think the parenthesis pair is correct. This specification of flatten is quite clear!
5) No opinion.
> COUNT(*) does not work
> -----------------------
>
> Key: PIG-812
> URL: https://issues.apache.org/jira/browse/PIG-812
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.2.0
> Reporter: Viraj Bhat
> Assignee: Benjamin Reed
> Fix For: 0.2.0
>
> Attachments: PIG-812.patch, PIG-812.patch, PIG-812.pdf, studenttab10k
>
>
> Pig script to count the number of rows in a studenttab10k file which contains 10k records.
> {code}
> studenttab = LOAD 'studenttab10k' AS (name:chararray, age:int,gpa:float);
> X2 = GROUP studenttab ALL;
> describe X2;
> Y2 = FOREACH X2 GENERATE COUNT(*);
> explain Y2;
> DUMP Y2;
> {code}
> returns the following error
> ================================================================
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias Y2
> Details at logfile: /homes/viraj/pig-svn/trunk/pig_1242783700970.log
> ================================================================
> If you look at the log file:
> ================================================================
> Caused by: java.lang.ClassCastException
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:76)
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:68)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:201)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:235)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:254)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:223)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:245)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:236)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:88)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> ================================================================
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-812) COUNT(*) does not work
Posted by "Benjamin Reed (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benjamin Reed updated PIG-812:
------------------------------
Status: Patch Available (was: Open)
> COUNT(*) does not work
> -----------------------
>
> Key: PIG-812
> URL: https://issues.apache.org/jira/browse/PIG-812
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.2.0
> Reporter: Viraj Bhat
> Assignee: Benjamin Reed
> Fix For: 0.2.0
>
> Attachments: PIG-812.patch, PIG-812.pdf, studenttab10k
>
>
> Pig script to count the number of rows in a studenttab10k file which contains 10k records.
> {code}
> studenttab = LOAD 'studenttab10k' AS (name:chararray, age:int,gpa:float);
> X2 = GROUP studenttab ALL;
> describe X2;
> Y2 = FOREACH X2 GENERATE COUNT(*);
> explain Y2;
> DUMP Y2;
> {code}
> returns the following error
> ================================================================
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias Y2
> Details at logfile: /homes/viraj/pig-svn/trunk/pig_1242783700970.log
> ================================================================
> If you look at the log file:
> ================================================================
> Caused by: java.lang.ClassCastException
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:76)
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:68)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:201)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:235)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:254)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:223)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:245)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:236)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:88)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> ================================================================
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-812) COUNT(*) does not work
Posted by "Benjamin Reed (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benjamin Reed updated PIG-812:
------------------------------
Attachment: (was: PIG-812.patch)
> COUNT(*) does not work
> -----------------------
>
> Key: PIG-812
> URL: https://issues.apache.org/jira/browse/PIG-812
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.2.0
> Reporter: Viraj Bhat
> Assignee: Benjamin Reed
> Fix For: 0.2.0
>
> Attachments: PIG-812.patch, PIG-812.pdf, studenttab10k
>
>
> Pig script to count the number of rows in a studenttab10k file which contains 10k records.
> {code}
> studenttab = LOAD 'studenttab10k' AS (name:chararray, age:int,gpa:float);
> X2 = GROUP studenttab ALL;
> describe X2;
> Y2 = FOREACH X2 GENERATE COUNT(*);
> explain Y2;
> DUMP Y2;
> {code}
> returns the following error
> ================================================================
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias Y2
> Details at logfile: /homes/viraj/pig-svn/trunk/pig_1242783700970.log
> ================================================================
> If you look at the log file:
> ================================================================
> Caused by: java.lang.ClassCastException
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:76)
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:68)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:201)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:235)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:254)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:223)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:245)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:236)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:88)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> ================================================================
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-812) COUNT(*) does not work
Posted by "Benjamin Reed (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benjamin Reed updated PIG-812:
------------------------------
Status: Open (was: Patch Available)
> COUNT(*) does not work
> -----------------------
>
> Key: PIG-812
> URL: https://issues.apache.org/jira/browse/PIG-812
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.2.0
> Reporter: Viraj Bhat
> Assignee: Benjamin Reed
> Fix For: 0.2.0
>
> Attachments: PIG-812.patch, PIG-812.pdf, studenttab10k
>
>
> Pig script to count the number of rows in a studenttab10k file which contains 10k records.
> {code}
> studenttab = LOAD 'studenttab10k' AS (name:chararray, age:int,gpa:float);
> X2 = GROUP studenttab ALL;
> describe X2;
> Y2 = FOREACH X2 GENERATE COUNT(*);
> explain Y2;
> DUMP Y2;
> {code}
> returns the following error
> ================================================================
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias Y2
> Details at logfile: /homes/viraj/pig-svn/trunk/pig_1242783700970.log
> ================================================================
> If you look at the log file:
> ================================================================
> Caused by: java.lang.ClassCastException
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:76)
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:68)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:201)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:235)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:254)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:223)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:245)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:236)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:88)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> ================================================================
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-812) COUNT(*) does not work
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12729472#action_12729472 ]
Hadoop QA commented on PIG-812:
-------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12413067/PIG-812.pdf
against trunk revision 792663.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
-1 patch. The patch command could not apply the patch.
Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-minerva.apache.org/119/console
This message is automatically generated.
> COUNT(*) does not work
> -----------------------
>
> Key: PIG-812
> URL: https://issues.apache.org/jira/browse/PIG-812
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.2.0
> Reporter: Viraj Bhat
> Assignee: Benjamin Reed
> Fix For: 0.2.0
>
> Attachments: PIG-812.patch, PIG-812.pdf, studenttab10k
>
>
> Pig script to count the number of rows in a studenttab10k file which contains 10k records.
> {code}
> studenttab = LOAD 'studenttab10k' AS (name:chararray, age:int,gpa:float);
> X2 = GROUP studenttab ALL;
> describe X2;
> Y2 = FOREACH X2 GENERATE COUNT(*);
> explain Y2;
> DUMP Y2;
> {code}
> returns the following error
> ================================================================
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias Y2
> Details at logfile: /homes/viraj/pig-svn/trunk/pig_1242783700970.log
> ================================================================
> If you look at the log file:
> ================================================================
> Caused by: java.lang.ClassCastException
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:76)
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:68)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:201)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:235)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:254)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:223)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:245)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:236)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:88)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> ================================================================
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-812) COUNT(*) does not work
Posted by "Benjamin Reed (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benjamin Reed updated PIG-812:
------------------------------
Status: Open (was: Patch Available)
> COUNT(*) does not work
> -----------------------
>
> Key: PIG-812
> URL: https://issues.apache.org/jira/browse/PIG-812
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.2.0
> Reporter: Viraj Bhat
> Assignee: Benjamin Reed
> Fix For: 0.2.0
>
> Attachments: PIG-812.patch, PIG-812.pdf, studenttab10k
>
>
> Pig script to count the number of rows in a studenttab10k file which contains 10k records.
> {code}
> studenttab = LOAD 'studenttab10k' AS (name:chararray, age:int,gpa:float);
> X2 = GROUP studenttab ALL;
> describe X2;
> Y2 = FOREACH X2 GENERATE COUNT(*);
> explain Y2;
> DUMP Y2;
> {code}
> returns the following error
> ================================================================
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias Y2
> Details at logfile: /homes/viraj/pig-svn/trunk/pig_1242783700970.log
> ================================================================
> If you look at the log file:
> ================================================================
> Caused by: java.lang.ClassCastException
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:76)
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:68)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:201)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:235)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:254)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:223)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:245)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:236)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:88)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> ================================================================
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-812) COUNT(*) does not work
Posted by "Viraj Bhat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Viraj Bhat updated PIG-812:
---------------------------
Attachment: studenttab10k
Input file
> COUNT(*) does not work
> -----------------------
>
> Key: PIG-812
> URL: https://issues.apache.org/jira/browse/PIG-812
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.2.0
> Reporter: Viraj Bhat
> Priority: Critical
> Fix For: 0.2.0
>
> Attachments: studenttab10k
>
>
> Pig script to count the number of rows in a studenttab10k file which contains 10k records.
> {code}
> studenttab = LOAD 'studenttab10k' AS (name:chararray, age:int,gpa:float);
> X2 = GROUP studenttab ALL;
> describe X2;
> Y2 = FOREACH X2 GENERATE COUNT(*);
> explain Y2;
> DUMP Y2;
> {code}
> returns the following error
> ================================================================
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias Y2
> Details at logfile: /homes/viraj/pig-svn/trunk/pig_1242783700970.log
> ================================================================
> If you look at the log file:
> ================================================================
> Caused by: java.lang.ClassCastException
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:76)
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:68)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:201)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:235)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:254)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:223)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:245)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:236)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:88)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> ================================================================
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-812) COUNT(*) does not work
Posted by "Benjamin Reed (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benjamin Reed updated PIG-812:
------------------------------
Status: Patch Available (was: Open)
> COUNT(*) does not work
> -----------------------
>
> Key: PIG-812
> URL: https://issues.apache.org/jira/browse/PIG-812
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.2.0
> Reporter: Viraj Bhat
> Assignee: Benjamin Reed
> Fix For: 0.2.0
>
> Attachments: PIG-812.patch, PIG-812.pdf, studenttab10k
>
>
> Pig script to count the number of rows in a studenttab10k file which contains 10k records.
> {code}
> studenttab = LOAD 'studenttab10k' AS (name:chararray, age:int,gpa:float);
> X2 = GROUP studenttab ALL;
> describe X2;
> Y2 = FOREACH X2 GENERATE COUNT(*);
> explain Y2;
> DUMP Y2;
> {code}
> returns the following error
> ================================================================
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias Y2
> Details at logfile: /homes/viraj/pig-svn/trunk/pig_1242783700970.log
> ================================================================
> If you look at the log file:
> ================================================================
> Caused by: java.lang.ClassCastException
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:76)
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:68)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:201)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:235)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:254)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:223)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:245)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:236)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:88)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> ================================================================
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-812) COUNT(*) does not work
Posted by "Benjamin Reed (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benjamin Reed updated PIG-812:
------------------------------
Priority: Major (was: Critical)
> COUNT(*) does not work
> -----------------------
>
> Key: PIG-812
> URL: https://issues.apache.org/jira/browse/PIG-812
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.2.0
> Reporter: Viraj Bhat
> Assignee: Benjamin Reed
> Fix For: 0.2.0
>
> Attachments: studenttab10k
>
>
> Pig script to count the number of rows in a studenttab10k file which contains 10k records.
> {code}
> studenttab = LOAD 'studenttab10k' AS (name:chararray, age:int,gpa:float);
> X2 = GROUP studenttab ALL;
> describe X2;
> Y2 = FOREACH X2 GENERATE COUNT(*);
> explain Y2;
> DUMP Y2;
> {code}
> returns the following error
> ================================================================
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias Y2
> Details at logfile: /homes/viraj/pig-svn/trunk/pig_1242783700970.log
> ================================================================
> If you look at the log file:
> ================================================================
> Caused by: java.lang.ClassCastException
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:76)
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:68)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:201)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:235)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:254)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:223)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:245)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:236)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:88)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> ================================================================
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-812) COUNT(*) does not work
Posted by "Benjamin Reed (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12744868#action_12744868 ]
Benjamin Reed commented on PIG-812:
-----------------------------------
@olga
1) can you suggest some text, i'm not a sql guru
2) this patch is focused on documenting * and flatten. i didn't really change the documentation for boolean, tuple, field, and general expression
3) you are saying that i should have 'key' in the description correct?
4) i don't think that is a typo. it is a tuple with one field that is a bag of tuples, when you flatten the bag field the individual tuples will pop out. can you suggest how to make it more clear?
5) it's easy enough to switch around, but it seemed that group was the more general term (if such a term could be used since they are aliases of each other :) where cogroup implies more than one. don't you think?
> COUNT(*) does not work
> -----------------------
>
> Key: PIG-812
> URL: https://issues.apache.org/jira/browse/PIG-812
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.2.0
> Reporter: Viraj Bhat
> Assignee: Benjamin Reed
> Fix For: 0.2.0
>
> Attachments: PIG-812.patch, PIG-812.patch, PIG-812.pdf, studenttab10k
>
>
> Pig script to count the number of rows in a studenttab10k file which contains 10k records.
> {code}
> studenttab = LOAD 'studenttab10k' AS (name:chararray, age:int,gpa:float);
> X2 = GROUP studenttab ALL;
> describe X2;
> Y2 = FOREACH X2 GENERATE COUNT(*);
> explain Y2;
> DUMP Y2;
> {code}
> returns the following error
> ================================================================
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias Y2
> Details at logfile: /homes/viraj/pig-svn/trunk/pig_1242783700970.log
> ================================================================
> If you look at the log file:
> ================================================================
> Caused by: java.lang.ClassCastException
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:76)
> at org.apache.pig.builtin.COUNT$Initial.exec(COUNT.java:68)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:201)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:235)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:254)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:231)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:223)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:245)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:236)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:88)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> ================================================================
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.