You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "John Amos (JIRA)" <ji...@apache.org> on 2011/06/10 18:26:58 UTC
[jira] [Created] (PIG-2118) ClassCastException when using group.$0
ClassCastException when using group.$0
--------------------------------------
Key: PIG-2118
URL: https://issues.apache.org/jira/browse/PIG-2118
Project: Pig
Issue Type: Bug
Affects Versions: 0.8.1
Environment: CentOS 5.6
Hadoop 0.20.2
Subversion https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 911707
Compiled by chrisdo on Fri Feb 19 08:07:34 UTC 2010
Apache Pig version 0.8.1 (r1094835)
compiled Apr 18 2011, 19:26:53
Also reproduced on:
CentOS 5.5
Apache Pig version 0.8.0-cdh3u0 (rexported)
compiled Mar 25 2011, 16:16:24
Hadoop 0.20.2-CDH3B4
Subversion -r 3aa7c91592ea1c53f3a913a581dbfcdfebe98bfe
Compiled by root on Mon Feb 21 17:31:12 EST 2011
>From source with checksum cd3f3059d069da355d3991b499d42c3b
Reporter: John Amos
Priority: Blocker
Create an input file named "key_only" that contains a single number.
key_only = LOAD 'key_only' AS (theKey: long);
A = GROUP key_only BY theKey;
B = FOREACH A GENERATE group.$0 AS theKey;
dump B;
Processing fails with the following error:
Backend error message
---------------------
java.lang.ClassCastException: java.lang.Long cannot be cast to org.apache.pig.data.Tuple
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:392)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:276)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:138)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:276)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:345)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:290)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.runPipeline(PigMapReduce.java:434)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.processOnePackageOutput(PigMapReduce.java:402)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:382)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:251)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:571)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:413)
at org.apache.hadoop.mapred.Child$4.run(Child.java:240)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
at org.apache.hadoop.mapred.Child.main(Child.java:234)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2118) ClassCastException when using
group.$0
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13047350#comment-13047350 ]
Daniel Dai commented on PIG-2118:
---------------------------------
Hi, John, the group key is "theKey", which is a long. group.$0 will throw a parser exception. Can you double check if this is the script you are running?
> ClassCastException when using group.$0
> --------------------------------------
>
> Key: PIG-2118
> URL: https://issues.apache.org/jira/browse/PIG-2118
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.1
> Environment: CentOS 5.6
> Hadoop 0.20.2
> Subversion https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 911707
> Compiled by chrisdo on Fri Feb 19 08:07:34 UTC 2010
> Apache Pig version 0.8.1 (r1094835)
> compiled Apr 18 2011, 19:26:53
> Also reproduced on:
> CentOS 5.5
> Apache Pig version 0.8.0-cdh3u0 (rexported)
> compiled Mar 25 2011, 16:16:24
> Hadoop 0.20.2-CDH3B4
> Subversion -r 3aa7c91592ea1c53f3a913a581dbfcdfebe98bfe
> Compiled by root on Mon Feb 21 17:31:12 EST 2011
> From source with checksum cd3f3059d069da355d3991b499d42c3b
> Reporter: John Amos
> Priority: Blocker
>
> Create an input file named "key_only" that contains a single number.
> key_only = LOAD 'key_only' AS (theKey: long);
> A = GROUP key_only BY theKey;
> B = FOREACH A GENERATE group.$0 AS theKey;
> dump B;
> Processing fails with the following error:
> Backend error message
> ---------------------
> java.lang.ClassCastException: java.lang.Long cannot be cast to org.apache.pig.data.Tuple
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:392)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:276)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:138)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:276)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:345)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:290)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.runPipeline(PigMapReduce.java:434)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.processOnePackageOutput(PigMapReduce.java:402)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:382)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:251)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:571)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:413)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:240)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
> at org.apache.hadoop.mapred.Child.main(Child.java:234)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2118) ClassCastException when using
group.$0
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13047412#comment-13047412 ]
Dmitriy V. Ryaboy commented on PIG-2118:
----------------------------------------
John -- if it's easier for your generator to always assume that group is a tuple, you can GROUP key_only by (theKey); (note the parenthesis).
> ClassCastException when using group.$0
> --------------------------------------
>
> Key: PIG-2118
> URL: https://issues.apache.org/jira/browse/PIG-2118
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.1
> Environment: CentOS 5.6
> Hadoop 0.20.2
> Subversion https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 911707
> Compiled by chrisdo on Fri Feb 19 08:07:34 UTC 2010
> Apache Pig version 0.8.1 (r1094835)
> compiled Apr 18 2011, 19:26:53
> Also reproduced on:
> CentOS 5.5
> Apache Pig version 0.8.0-cdh3u0 (rexported)
> compiled Mar 25 2011, 16:16:24
> Hadoop 0.20.2-CDH3B4
> Subversion -r 3aa7c91592ea1c53f3a913a581dbfcdfebe98bfe
> Compiled by root on Mon Feb 21 17:31:12 EST 2011
> From source with checksum cd3f3059d069da355d3991b499d42c3b
> Reporter: John Amos
> Priority: Blocker
>
> Create an input file named "key_only" that contains a single number.
> key_only = LOAD 'key_only' AS (theKey: long);
> A = GROUP key_only BY theKey;
> B = FOREACH A GENERATE group.$0 AS theKey;
> dump B;
> Processing fails with the following error:
> Backend error message
> ---------------------
> java.lang.ClassCastException: java.lang.Long cannot be cast to org.apache.pig.data.Tuple
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:392)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:276)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:138)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:276)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:345)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:290)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.runPipeline(PigMapReduce.java:434)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.processOnePackageOutput(PigMapReduce.java:402)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:382)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:251)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:571)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:413)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:240)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
> at org.apache.hadoop.mapred.Child.main(Child.java:234)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2118) ClassCastException when using
group.$0
Posted by "Thejas M Nair (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13047388#comment-13047388 ]
Thejas M Nair commented on PIG-2118:
------------------------------------
bq. If this syntax is invalid shouldn't it fail during parse time instead of throwing a ClassCastException at run time?
Yes, it should give a parse error. The bug been fixed for 0.9 ( PIG-1281).
In 0.9 -
{code}
grunt> key_only = LOAD 'key_only' AS (theKey: long);
grunt> A = GROUP key_only BY theKey;
grunt> B = FOREACH A GENERATE group.$0 AS theKey;
2011-06-10 12:19:47,849 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1129:
<line 7, column 28> Referring to column(s) within a column of type long is not allowed
Details at logfile: /Users/tejas/pig_sedes_long/trunk/pig_1307671501595.log
{code}
> ClassCastException when using group.$0
> --------------------------------------
>
> Key: PIG-2118
> URL: https://issues.apache.org/jira/browse/PIG-2118
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.1
> Environment: CentOS 5.6
> Hadoop 0.20.2
> Subversion https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 911707
> Compiled by chrisdo on Fri Feb 19 08:07:34 UTC 2010
> Apache Pig version 0.8.1 (r1094835)
> compiled Apr 18 2011, 19:26:53
> Also reproduced on:
> CentOS 5.5
> Apache Pig version 0.8.0-cdh3u0 (rexported)
> compiled Mar 25 2011, 16:16:24
> Hadoop 0.20.2-CDH3B4
> Subversion -r 3aa7c91592ea1c53f3a913a581dbfcdfebe98bfe
> Compiled by root on Mon Feb 21 17:31:12 EST 2011
> From source with checksum cd3f3059d069da355d3991b499d42c3b
> Reporter: John Amos
> Priority: Blocker
>
> Create an input file named "key_only" that contains a single number.
> key_only = LOAD 'key_only' AS (theKey: long);
> A = GROUP key_only BY theKey;
> B = FOREACH A GENERATE group.$0 AS theKey;
> dump B;
> Processing fails with the following error:
> Backend error message
> ---------------------
> java.lang.ClassCastException: java.lang.Long cannot be cast to org.apache.pig.data.Tuple
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:392)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:276)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:138)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:276)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:345)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:290)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.runPipeline(PigMapReduce.java:434)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.processOnePackageOutput(PigMapReduce.java:402)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:382)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:251)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:571)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:413)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:240)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
> at org.apache.hadoop.mapred.Child.main(Child.java:234)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (PIG-2118) ClassCastException when using group.$0
Posted by "Dmitriy V. Ryaboy (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy resolved PIG-2118.
------------------------------------
Resolution: Not A Problem
This is partially not a problem (user was auto-generating incorrect syntax), and partially a ux issue fixed in a separate ticket (user should've been told at compile time that he was using incorrect syntax). Closing.
> ClassCastException when using group.$0
> --------------------------------------
>
> Key: PIG-2118
> URL: https://issues.apache.org/jira/browse/PIG-2118
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.1
> Environment: CentOS 5.6
> Hadoop 0.20.2
> Subversion https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 911707
> Compiled by chrisdo on Fri Feb 19 08:07:34 UTC 2010
> Apache Pig version 0.8.1 (r1094835)
> compiled Apr 18 2011, 19:26:53
> Also reproduced on:
> CentOS 5.5
> Apache Pig version 0.8.0-cdh3u0 (rexported)
> compiled Mar 25 2011, 16:16:24
> Hadoop 0.20.2-CDH3B4
> Subversion -r 3aa7c91592ea1c53f3a913a581dbfcdfebe98bfe
> Compiled by root on Mon Feb 21 17:31:12 EST 2011
> From source with checksum cd3f3059d069da355d3991b499d42c3b
> Reporter: John Amos
> Priority: Blocker
>
> Create an input file named "key_only" that contains a single number.
> key_only = LOAD 'key_only' AS (theKey: long);
> A = GROUP key_only BY theKey;
> B = FOREACH A GENERATE group.$0 AS theKey;
> dump B;
> Processing fails with the following error:
> Backend error message
> ---------------------
> java.lang.ClassCastException: java.lang.Long cannot be cast to org.apache.pig.data.Tuple
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:392)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:276)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:138)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:276)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:345)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:290)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.runPipeline(PigMapReduce.java:434)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.processOnePackageOutput(PigMapReduce.java:402)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:382)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:251)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:571)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:413)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:240)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
> at org.apache.hadoop.mapred.Child.main(Child.java:234)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2118) ClassCastException when using
group.$0
Posted by "John Amos (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13047382#comment-13047382 ]
John Amos commented on PIG-2118:
--------------------------------
I use group.$0 because I built a Pig Latin generator and it was easier to implement the generator that way. If this syntax is invalid shouldn't it fail during parse time instead of throwing a ClassCastException at run time?
> ClassCastException when using group.$0
> --------------------------------------
>
> Key: PIG-2118
> URL: https://issues.apache.org/jira/browse/PIG-2118
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.1
> Environment: CentOS 5.6
> Hadoop 0.20.2
> Subversion https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 911707
> Compiled by chrisdo on Fri Feb 19 08:07:34 UTC 2010
> Apache Pig version 0.8.1 (r1094835)
> compiled Apr 18 2011, 19:26:53
> Also reproduced on:
> CentOS 5.5
> Apache Pig version 0.8.0-cdh3u0 (rexported)
> compiled Mar 25 2011, 16:16:24
> Hadoop 0.20.2-CDH3B4
> Subversion -r 3aa7c91592ea1c53f3a913a581dbfcdfebe98bfe
> Compiled by root on Mon Feb 21 17:31:12 EST 2011
> From source with checksum cd3f3059d069da355d3991b499d42c3b
> Reporter: John Amos
> Priority: Blocker
>
> Create an input file named "key_only" that contains a single number.
> key_only = LOAD 'key_only' AS (theKey: long);
> A = GROUP key_only BY theKey;
> B = FOREACH A GENERATE group.$0 AS theKey;
> dump B;
> Processing fails with the following error:
> Backend error message
> ---------------------
> java.lang.ClassCastException: java.lang.Long cannot be cast to org.apache.pig.data.Tuple
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:392)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:276)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:138)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:276)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:345)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:290)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.runPipeline(PigMapReduce.java:434)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.processOnePackageOutput(PigMapReduce.java:402)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:382)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:251)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:571)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:413)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:240)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
> at org.apache.hadoop.mapred.Child.main(Child.java:234)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2118) ClassCastException when using
group.$0
Posted by "John Amos (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13047372#comment-13047372 ]
John Amos commented on PIG-2118:
--------------------------------
Hi Daniel, I copied and pasted the script into grunt in two different environments (see "Environment" section above for details) and they both fail with the same error.
> ClassCastException when using group.$0
> --------------------------------------
>
> Key: PIG-2118
> URL: https://issues.apache.org/jira/browse/PIG-2118
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.1
> Environment: CentOS 5.6
> Hadoop 0.20.2
> Subversion https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 911707
> Compiled by chrisdo on Fri Feb 19 08:07:34 UTC 2010
> Apache Pig version 0.8.1 (r1094835)
> compiled Apr 18 2011, 19:26:53
> Also reproduced on:
> CentOS 5.5
> Apache Pig version 0.8.0-cdh3u0 (rexported)
> compiled Mar 25 2011, 16:16:24
> Hadoop 0.20.2-CDH3B4
> Subversion -r 3aa7c91592ea1c53f3a913a581dbfcdfebe98bfe
> Compiled by root on Mon Feb 21 17:31:12 EST 2011
> From source with checksum cd3f3059d069da355d3991b499d42c3b
> Reporter: John Amos
> Priority: Blocker
>
> Create an input file named "key_only" that contains a single number.
> key_only = LOAD 'key_only' AS (theKey: long);
> A = GROUP key_only BY theKey;
> B = FOREACH A GENERATE group.$0 AS theKey;
> dump B;
> Processing fails with the following error:
> Backend error message
> ---------------------
> java.lang.ClassCastException: java.lang.Long cannot be cast to org.apache.pig.data.Tuple
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:392)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:276)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:138)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:276)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:345)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:290)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.runPipeline(PigMapReduce.java:434)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.processOnePackageOutput(PigMapReduce.java:402)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:382)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:251)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:571)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:413)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:240)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
> at org.apache.hadoop.mapred.Child.main(Child.java:234)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2118) ClassCastException when using
group.$0
Posted by "John Amos (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13047281#comment-13047281 ]
John Amos commented on PIG-2118:
--------------------------------
Work-around:
Insert the following at POProject.java:392
if (!(ret instanceof Tuple)) {
ret = tupleFactory.newTuple(ret);
}
> ClassCastException when using group.$0
> --------------------------------------
>
> Key: PIG-2118
> URL: https://issues.apache.org/jira/browse/PIG-2118
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.1
> Environment: CentOS 5.6
> Hadoop 0.20.2
> Subversion https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 911707
> Compiled by chrisdo on Fri Feb 19 08:07:34 UTC 2010
> Apache Pig version 0.8.1 (r1094835)
> compiled Apr 18 2011, 19:26:53
> Also reproduced on:
> CentOS 5.5
> Apache Pig version 0.8.0-cdh3u0 (rexported)
> compiled Mar 25 2011, 16:16:24
> Hadoop 0.20.2-CDH3B4
> Subversion -r 3aa7c91592ea1c53f3a913a581dbfcdfebe98bfe
> Compiled by root on Mon Feb 21 17:31:12 EST 2011
> From source with checksum cd3f3059d069da355d3991b499d42c3b
> Reporter: John Amos
> Priority: Blocker
>
> Create an input file named "key_only" that contains a single number.
> key_only = LOAD 'key_only' AS (theKey: long);
> A = GROUP key_only BY theKey;
> B = FOREACH A GENERATE group.$0 AS theKey;
> dump B;
> Processing fails with the following error:
> Backend error message
> ---------------------
> java.lang.ClassCastException: java.lang.Long cannot be cast to org.apache.pig.data.Tuple
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:392)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:276)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:138)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:276)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:345)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:290)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.runPipeline(PigMapReduce.java:434)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.processOnePackageOutput(PigMapReduce.java:402)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:382)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:251)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:571)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:413)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:240)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
> at org.apache.hadoop.mapred.Child.main(Child.java:234)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2118) ClassCastException when using
group.$0
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13047377#comment-13047377 ]
Daniel Dai commented on PIG-2118:
---------------------------------
Why you refer to group.$0? Seems "group" should be the column "theKey" you want to refer.
> ClassCastException when using group.$0
> --------------------------------------
>
> Key: PIG-2118
> URL: https://issues.apache.org/jira/browse/PIG-2118
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.8.1
> Environment: CentOS 5.6
> Hadoop 0.20.2
> Subversion https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 911707
> Compiled by chrisdo on Fri Feb 19 08:07:34 UTC 2010
> Apache Pig version 0.8.1 (r1094835)
> compiled Apr 18 2011, 19:26:53
> Also reproduced on:
> CentOS 5.5
> Apache Pig version 0.8.0-cdh3u0 (rexported)
> compiled Mar 25 2011, 16:16:24
> Hadoop 0.20.2-CDH3B4
> Subversion -r 3aa7c91592ea1c53f3a913a581dbfcdfebe98bfe
> Compiled by root on Mon Feb 21 17:31:12 EST 2011
> From source with checksum cd3f3059d069da355d3991b499d42c3b
> Reporter: John Amos
> Priority: Blocker
>
> Create an input file named "key_only" that contains a single number.
> key_only = LOAD 'key_only' AS (theKey: long);
> A = GROUP key_only BY theKey;
> B = FOREACH A GENERATE group.$0 AS theKey;
> dump B;
> Processing fails with the following error:
> Backend error message
> ---------------------
> java.lang.ClassCastException: java.lang.Long cannot be cast to org.apache.pig.data.Tuple
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:392)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:276)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:138)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:276)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:345)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:290)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.runPipeline(PigMapReduce.java:434)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.processOnePackageOutput(PigMapReduce.java:402)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:382)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.reduce(PigMapReduce.java:251)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:571)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:413)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:240)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
> at org.apache.hadoop.mapred.Child.main(Child.java:234)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira