You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Zheng Shao (JIRA)" <ji...@apache.org> on 2008/12/08 22:42:44 UTC

[jira] Commented: (HIVE-136) SerDe should escape some special characters

    [ https://issues.apache.org/jira/browse/HIVE-136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654585#action_12654585 ] 

Zheng Shao commented on HIVE-136:
---------------------------------

The problem breaks down to 3 items:

1. When loading the data, the user would be responsible for providing data in the expected escaped format. For example, in order to represent in the text file one field containing a carriage return, it should be written as "\n". Sequence file does not have to escape "\n" but that will be recommended. Both file formats would need to escape column/item/key separator.

2. When the serde is reading the data, it should unescape the data. This means string.split need to be replaced by a scan, which will have a little performance loss.

3. When the serde is writing the data, it should escape the data. This also means a scan of the string and conversion to a new string, which also have a little performance loss.


> SerDe should escape some special characters
> -------------------------------------------
>
>                 Key: HIVE-136
>                 URL: https://issues.apache.org/jira/browse/HIVE-136
>             Project: Hadoop Hive
>          Issue Type: Bug
>            Reporter: Zheng Shao
>
> MetadataTypedColumnsetSerDe and DynamicSerDe should escape some special characters like '\n' or the column/item/key separator.
> Otherwise the data will look corrupted.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.