You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Thejas M Nair (JIRA)" <ji...@apache.org> on 2009/03/03 23:30:56 UTC
[jira] Created: (PIG-696) Fatal error produced when malformed
scalar types within complex type is converted to given type
Fatal error produced when malformed scalar types within complex type is converted to given type
-----------------------------------------------------------------------------------------------
Key: PIG-696
URL: https://issues.apache.org/jira/browse/PIG-696
Project: Pig
Issue Type: Bug
Reporter: Thejas M Nair
Instead of fatal error, the failed conversions should result in null values.
Example -
grunt > cat cbag3.dat
{(asdf)}
{(2344)}
{(2344}
{(323423423423434)}
{(323423423423434L)}
{(asdff)}
grunt> A = load 'cbag3.dat' as (f1:bag{t:tuple(i:int)}); B = foreach A generate flatten(f1); C = foreach B generate $0 + 1; dump C;
2009-03-03 14:25:19,604 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete
2009-03-03 14:25:44,628 [main] ERROR org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Map reduce job failed
2009-03-03 14:25:44,642 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2043: Unexpected error during execution.
Details at logfile: /d1/tejas/pig_1236118410343.log
tail /d1/tejas/pig_1236118410343.log
Caused by: java.lang.ClassCastException
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Add.getNext(Add.java:110)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:260)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:198)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:217)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:208)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.map(PigMapOnly.java:65)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
The 'conversion' of scalar types in complex types is happening in the physicaloperators, and not in the loaders. The expressions (such as Add in example) attempts to cast input to given type, and ClassCastException is thrown when conversion fails.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-696) Fatal error produced when malformed
scalar types within complex type is converted to given type
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich updated PIG-696:
-------------------------------
Fix Version/s: 0.9.0
> Fatal error produced when malformed scalar types within complex type is converted to given type
> -----------------------------------------------------------------------------------------------
>
> Key: PIG-696
> URL: https://issues.apache.org/jira/browse/PIG-696
> Project: Pig
> Issue Type: Bug
> Reporter: Thejas M Nair
> Fix For: 0.9.0
>
>
> Instead of fatal error, the failed conversions should result in null values.
> Example -
> grunt > cat cbag3.dat
> {(asdf)}
> {(2344)}
> {(2344}
> {(323423423423434)}
> {(323423423423434L)}
> {(asdff)}
> grunt> A = load 'cbag3.dat' as (f1:bag{t:tuple(i:int)}); B = foreach A generate flatten(f1); C = foreach B generate $0 + 1; dump C;
> 2009-03-03 14:25:19,604 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete
> 2009-03-03 14:25:44,628 [main] ERROR org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Map reduce job failed
> 2009-03-03 14:25:44,642 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2043: Unexpected error during execution.
> Details at logfile: /d1/tejas/pig_1236118410343.log
> tail /d1/tejas/pig_1236118410343.log
> Caused by: java.lang.ClassCastException
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Add.getNext(Add.java:110)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:260)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:198)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:217)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:208)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.map(PigMapOnly.java:65)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> The 'conversion' of scalar types in complex types is happening in the physicaloperators, and not in the loaders. The expressions (such as Add in example) attempts to cast input to given type, and ClassCastException is thrown when conversion fails.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (PIG-696) Fatal error produced when malformed
scalar types within complex type is converted to given type
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Gates reassigned PIG-696:
------------------------------
Assignee: Alan Gates
> Fatal error produced when malformed scalar types within complex type is converted to given type
> -----------------------------------------------------------------------------------------------
>
> Key: PIG-696
> URL: https://issues.apache.org/jira/browse/PIG-696
> Project: Pig
> Issue Type: Bug
> Reporter: Thejas M Nair
> Assignee: Alan Gates
> Fix For: 0.9.0
>
>
> Instead of fatal error, the failed conversions should result in null values.
> Example -
> grunt > cat cbag3.dat
> {(asdf)}
> {(2344)}
> {(2344}
> {(323423423423434)}
> {(323423423423434L)}
> {(asdff)}
> grunt> A = load 'cbag3.dat' as (f1:bag{t:tuple(i:int)}); B = foreach A generate flatten(f1); C = foreach B generate $0 + 1; dump C;
> 2009-03-03 14:25:19,604 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete
> 2009-03-03 14:25:44,628 [main] ERROR org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Map reduce job failed
> 2009-03-03 14:25:44,642 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2043: Unexpected error during execution.
> Details at logfile: /d1/tejas/pig_1236118410343.log
> tail /d1/tejas/pig_1236118410343.log
> Caused by: java.lang.ClassCastException
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Add.getNext(Add.java:110)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:260)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:198)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:217)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:208)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.map(PigMapOnly.java:65)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> The 'conversion' of scalar types in complex types is happening in the physicaloperators, and not in the loaders. The expressions (such as Add in example) attempts to cast input to given type, and ClassCastException is thrown when conversion fails.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (PIG-696) Fatal error produced when malformed
scalar types within complex type is converted to given type
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai resolved PIG-696.
----------------------------
Resolution: Fixed
Verified already fixed in trunk.
> Fatal error produced when malformed scalar types within complex type is converted to given type
> -----------------------------------------------------------------------------------------------
>
> Key: PIG-696
> URL: https://issues.apache.org/jira/browse/PIG-696
> Project: Pig
> Issue Type: Bug
> Reporter: Thejas M Nair
> Assignee: Alan Gates
> Fix For: 0.9.0
>
>
> Instead of fatal error, the failed conversions should result in null values.
> Example -
> grunt > cat cbag3.dat
> {(asdf)}
> {(2344)}
> {(2344}
> {(323423423423434)}
> {(323423423423434L)}
> {(asdff)}
> grunt> A = load 'cbag3.dat' as (f1:bag{t:tuple(i:int)}); B = foreach A generate flatten(f1); C = foreach B generate $0 + 1; dump C;
> 2009-03-03 14:25:19,604 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete
> 2009-03-03 14:25:44,628 [main] ERROR org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Map reduce job failed
> 2009-03-03 14:25:44,642 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2043: Unexpected error during execution.
> Details at logfile: /d1/tejas/pig_1236118410343.log
> tail /d1/tejas/pig_1236118410343.log
> Caused by: java.lang.ClassCastException
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Add.getNext(Add.java:110)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:260)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:198)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:217)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:208)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.map(PigMapOnly.java:65)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> The 'conversion' of scalar types in complex types is happening in the physicaloperators, and not in the loaders. The expressions (such as Add in example) attempts to cast input to given type, and ClassCastException is thrown when conversion fails.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.