You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Jeremy Karn (Updated) (JIRA)" <ji...@apache.org> on 2011/11/04 03:37:32 UTC
[jira] [Updated] (PIG-2351) Casting Exception in Illustrate for
POForEach
[ https://issues.apache.org/jira/browse/PIG-2351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jeremy Karn updated PIG-2351:
-----------------------------
Attachment: PIG-2351-test.patch
Test case exposing bug.
> Casting Exception in Illustrate for POForEach
> ---------------------------------------------
>
> Key: PIG-2351
> URL: https://issues.apache.org/jira/browse/PIG-2351
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.0, 0.10
> Reporter: Jeremy Karn
> Attachments: PIG-2351-test.patch
>
>
> When running illustrate on this script:
> {code}
> S1 = LOAD 'excite.log.bz2' AS (user_id:chararray, timestamp:chararray, query:chararray);
> S2 = LOAD 'excite.log.bz2' AS (user_id:chararray, timestamp:chararray, query:chararray);
> C = cogroup S1 BY user_id INNER, S2 by user_id INNER;
> D = foreach C
> generate group,
> flatten(S1.timestamp) as t1,
> flatten(S2.timestamp) as t2;
> STORE D INTO 'output';
> {code}
> I get this exception:
> {code}
> 2011-11-03 20:49:13,577 [main] ERROR org.apache.pig.PigServer - java.lang.ClassCastException: java.lang.String cannot be cast to org.apache.pig.data.Tuple
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.illustratorMarkup2(POForEach.java:683)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.createTuple(POForEach.java:458)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:404)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POOptimizedForEach.getNext(POOptimizedForEach.java:124)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POJoinPackage.getNext(POJoinPackage.java:222)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.processOnePackageOutput(PigGenericMapReduce.java:416)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:399)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:1)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
> at org.apache.pig.pen.LocalMapReduceSimulator.launchPig(LocalMapReduceSimulator.java:215)
> ...
> {code}
> The casting error itself is easy to fix (by checking for type) - but that results in the tracked lineages missing tuples. The method POForEach.illustratorMarkup2 seems to be based on the incorrect assumption that Object[] is an array of tuples instead of an array of fields for the output tuple and its not clear to me if this function needs to be modified or if there's a bigger problem somewhere else in the code.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira