[jira] [Commented] (TRAFODION-1911) A special mode to be more tolerant during bulkloading and accessing HDFS raw data

["liu ming (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TRAFODION-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355184#comment-15355184 ] 

liu ming commented on TRAFODION-1911:
-------------------------------------

up to now, most requirements meet. Close this for now. If new requirements arrive, reopen it.

> A special mode to be more tolerant during bulkloading and accessing HDFS raw data
> ---------------------------------------------------------------------------------
>
>                 Key: TRAFODION-1911
>                 URL: https://issues.apache.org/jira/browse/TRAFODION-1911
>             Project: Apache Trafodion
>          Issue Type: Umbrella
>            Reporter: liu ming
>            Assignee: liu ming
>
> Raw data in HDFS sometimes are not well formatted or contains bad values. Trafodion bulkloader will complain error or not work well by default once that is the case.
> It will be valuable to let Trafodion work in a special mode to tolerate bad raw data in the HDFS. A not completed list below:
> 1.	Automatically remove ‘\r’ when it is used as ‘\r\n’ the DOS format line delimiter. 
> 2.	Donot raise SQL error, but convert bad data into null automatically, and still be able to log this into error log files when required, don’t make this change silent, and make this action traceable. 
> 3.	Allow ‘\n’ in data field even ‘\n’ is the line terminator
> 4.	Auto truncate overflowed string, log it into the error log file, in order to make it traceable.
> 5.	Tolerate ‘\0’ in source data
> There may be more, this is an umbrella JIRA to track all related sub tasks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)