You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Viraj Bhat (JIRA)" <ji...@apache.org> on 2009/01/01 00:16:44 UTC
[jira] Created: (PIG-595) Use of Combiner causes
java.lang.ClassCastException in ForEach
Use of Combiner causes java.lang.ClassCastException in ForEach
--------------------------------------------------------------
Key: PIG-595
URL: https://issues.apache.org/jira/browse/PIG-595
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: types_branch
Reporter: Viraj Bhat
Fix For: types_branch
Attachments: querypairs.txt
The following Pig script causes a ClassCastException when QueryPairs is used in the ForEach statement. This is due to the use of the combiner.
{code}
QueryPairs = load 'querypairs.txt' using PigStorage() as ( q1: chararray, q2: chararray );
describe QueryPairs;
QueryPairsGrouped = group QueryPairs by ( q1 );
describe QueryPairsGrouped;
QueryGroups = foreach QueryPairsGrouped generate
group as q1,
COUNT(QueryPairs) as paircount,
QueryPairs as QueryPairs;
describe QueryGroups;
dump QueryGroups;
{code}
=================================================================================================================
2008-12-31 15:01:48,713 [main] ERROR org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher - Error message from task (map) task_200812151518_4922_m_000000java.lang.ClassCastException: org.apache.pig.data.DefaultDataBag cannot be cast to org.apache.pig.data.Tuple
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POCombinerPackage.getNext(POCombinerPackage.java:122)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigCombiner$Combine.processOnePackageOutput(PigCombiner.java:152)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigCombiner$Combine.reduce(PigCombiner.java:143)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigCombiner$Combine.reduce(PigCombiner.java:57)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.combineAndSpill(MapTask.java:904)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:785)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:698)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:228)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
=================================================================================================================
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (PIG-595) Use of Combiner causes
java.lang.ClassCastException in ForEach
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich resolved PIG-595.
--------------------------------
Resolution: Fixed
This bug is duplicate of https://issues.apache.org/jira/browse/PIG-746
> Use of Combiner causes java.lang.ClassCastException in ForEach
> --------------------------------------------------------------
>
> Key: PIG-595
> URL: https://issues.apache.org/jira/browse/PIG-595
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.2.0
> Reporter: Viraj Bhat
> Attachments: querypairs.txt
>
>
> The following Pig script causes a ClassCastException when QueryPairs is used in the ForEach statement. This is due to the use of the combiner.
> {code}
> QueryPairs = load 'querypairs.txt' using PigStorage() as ( q1: chararray, q2: chararray );
> describe QueryPairs;
> QueryPairsGrouped = group QueryPairs by ( q1 );
> describe QueryPairsGrouped;
> QueryGroups = foreach QueryPairsGrouped generate
> group as q1,
> COUNT(QueryPairs) as paircount,
> QueryPairs as QueryPairs;
> describe QueryGroups;
> dump QueryGroups;
> {code}
> =================================================================================================================
> 2008-12-31 15:01:48,713 [main] ERROR org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher - Error message from task (map) task_200812151518_4922_m_000000java.lang.ClassCastException: org.apache.pig.data.DefaultDataBag cannot be cast to org.apache.pig.data.Tuple
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POCombinerPackage.getNext(POCombinerPackage.java:122)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigCombiner$Combine.processOnePackageOutput(PigCombiner.java:152)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigCombiner$Combine.reduce(PigCombiner.java:143)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigCombiner$Combine.reduce(PigCombiner.java:57)
> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.combineAndSpill(MapTask.java:904)
> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:785)
> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:698)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:228)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
> =================================================================================================================
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-595) Use of Combiner causes
java.lang.ClassCastException in ForEach
Posted by "Viraj Bhat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Viraj Bhat updated PIG-595:
---------------------------
Attachment: querypairs.txt
Input file
> Use of Combiner causes java.lang.ClassCastException in ForEach
> --------------------------------------------------------------
>
> Key: PIG-595
> URL: https://issues.apache.org/jira/browse/PIG-595
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: types_branch
> Reporter: Viraj Bhat
> Fix For: types_branch
>
> Attachments: querypairs.txt
>
>
> The following Pig script causes a ClassCastException when QueryPairs is used in the ForEach statement. This is due to the use of the combiner.
> {code}
> QueryPairs = load 'querypairs.txt' using PigStorage() as ( q1: chararray, q2: chararray );
> describe QueryPairs;
> QueryPairsGrouped = group QueryPairs by ( q1 );
> describe QueryPairsGrouped;
> QueryGroups = foreach QueryPairsGrouped generate
> group as q1,
> COUNT(QueryPairs) as paircount,
> QueryPairs as QueryPairs;
> describe QueryGroups;
> dump QueryGroups;
> {code}
> =================================================================================================================
> 2008-12-31 15:01:48,713 [main] ERROR org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher - Error message from task (map) task_200812151518_4922_m_000000java.lang.ClassCastException: org.apache.pig.data.DefaultDataBag cannot be cast to org.apache.pig.data.Tuple
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POCombinerPackage.getNext(POCombinerPackage.java:122)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigCombiner$Combine.processOnePackageOutput(PigCombiner.java:152)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigCombiner$Combine.reduce(PigCombiner.java:143)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigCombiner$Combine.reduce(PigCombiner.java:57)
> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.combineAndSpill(MapTask.java:904)
> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:785)
> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:698)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:228)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
> =================================================================================================================
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (PIG-595) Use of Combiner causes
java.lang.ClassCastException in ForEach
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Gates reassigned PIG-595:
------------------------------
Assignee: Viraj Bhat
> Use of Combiner causes java.lang.ClassCastException in ForEach
> --------------------------------------------------------------
>
> Key: PIG-595
> URL: https://issues.apache.org/jira/browse/PIG-595
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.2.0
> Reporter: Viraj Bhat
> Assignee: Viraj Bhat
> Attachments: querypairs.txt
>
>
> The following Pig script causes a ClassCastException when QueryPairs is used in the ForEach statement. This is due to the use of the combiner.
> {code}
> QueryPairs = load 'querypairs.txt' using PigStorage() as ( q1: chararray, q2: chararray );
> describe QueryPairs;
> QueryPairsGrouped = group QueryPairs by ( q1 );
> describe QueryPairsGrouped;
> QueryGroups = foreach QueryPairsGrouped generate
> group as q1,
> COUNT(QueryPairs) as paircount,
> QueryPairs as QueryPairs;
> describe QueryGroups;
> dump QueryGroups;
> {code}
> =================================================================================================================
> 2008-12-31 15:01:48,713 [main] ERROR org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher - Error message from task (map) task_200812151518_4922_m_000000java.lang.ClassCastException: org.apache.pig.data.DefaultDataBag cannot be cast to org.apache.pig.data.Tuple
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POCombinerPackage.getNext(POCombinerPackage.java:122)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigCombiner$Combine.processOnePackageOutput(PigCombiner.java:152)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigCombiner$Combine.reduce(PigCombiner.java:143)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigCombiner$Combine.reduce(PigCombiner.java:57)
> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.combineAndSpill(MapTask.java:904)
> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:785)
> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:698)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:228)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)
> =================================================================================================================
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.