You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Santhosh Srinivasan (JIRA)" <ji...@apache.org> on 2009/02/20 19:29:02 UTC
[jira] Issue Comment Edited: (PIG-544) Utf8StorageConverter.java
does not always produce NULLs when data is malformed
[ https://issues.apache.org/jira/browse/PIG-544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12650810#action_12650810 ]
sms edited comment on PIG-544 at 2/20/09 10:27 AM:
-------------------------------------------------------------------
Another use case where scalars also generate errors:
{code}
grunt> a = load 'student_tab.data';
grunt> store a into 'student_tab.bin' using BinStorage();
grunt> a = load 'student_tab.bin' using BinStorage() as (name: int, age: int, gpa: float);
grunt> dump a;
2008-11-25 16:02:40,986 [main] ERROR org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher - Error message from task (map) task_200809241441_24635_m_000000java.lang.RuntimeException : Unexpected data type 74 found in stream.
at org.apache.pig.data.DataReaderWriter.readDatum(DataReaderWriter.java:115)
at org.apache.pig.builtin.BinStorage.bytesToInteger(BinStorage.java:169)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast.getNext(POCast.java:143)
....
{code}
was (Author: sms):
Another use case where scalars also generate errors:
{code}
grunt> a = load 'student_tab.data';
grunt> store a into 'student_tab.bin' using BinStorage();
grunt> a = load 'student_tab.bin' using BinStorage() as (name: int, age: int, gpa: float);
grunt> dump a;
2008-11-25 16:02:40,986 [main] ERROR org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher - Error message from task (map) task_200809241441_24635_m_000000java.lang.RuntimeException : Unexpected data type 74 found in stream. at org.apache.pig.data.DataReaderWriter.readDatum(DataReaderWriter.java:115) at org.apache.pig.builtin.BinStorage.bytesToInteger(BinStorage.java:169) at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast.getNext(POCast.java:143)
....
{code}
> Utf8StorageConverter.java does not always produce NULLs when data is malformed
> ------------------------------------------------------------------------------
>
> Key: PIG-544
> URL: https://issues.apache.org/jira/browse/PIG-544
> Project: Pig
> Issue Type: Bug
> Reporter: Olga Natkovich
> Assignee: Thejas M Nair
>
> It does so for scalar types but not for complext types and not for the fields inside of the complext types.
> This is because it uses different code to parse scalar types by themselves and scalar types inside of a complex type. It should really use the same (its own code to do so.)
> The code it is currently uses, is inside of TextDataParser.jjt and is also used to parse constants so we need to be careful if we want to make changes to it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.