You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by "Qian Xu (JIRA)" <ji...@apache.org> on 2015/01/09 06:06:34 UTC
[jira] [Updated] (SQOOP-1988) Sqoop2: isNull handling should be
moved to CSVIntermediateDataFormat
[ https://issues.apache.org/jira/browse/SQOOP-1988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Qian Xu updated SQOOP-1988:
---------------------------
Description:
Matcher should only match fields, not do any value modification (except when fallback to null ), Given argument {{fields}} should have converted null values.
Currently it will convert {{null}} {{"NULL"}} {{"null"}} {{"'null'"}} and {{""}} to null. There are two questions here:
1. They are CSV relevant. As we do not have a knowledge of what intermediate data format is used, why we do the check and convert here.
2. {{"'null'"}} (double quote and single quote), it is a valid string represents single quote null single quote. It will be converted to null. Is this valid?
I'd propose to remove it. And it will have performance advance. Thanks [~jerrychenhf]
was:
Matcher should only match fields, not do any value modification (except when fallback to null ), Given argument {{fields}} should have converted null values.
Currently it will convert {{null}} {{"NULL"}} {{"null"}} {{"'null'"}} and {{""}} to null. They are CSV relevant. As we do not have a knowledge of what intermediate data format is used, I'd remove the check and convert here.
Another benefit is performance gain. Thanks [~jerrychenhf]
> Sqoop2: isNull handling should be moved to CSVIntermediateDataFormat
> --------------------------------------------------------------------
>
> Key: SQOOP-1988
> URL: https://issues.apache.org/jira/browse/SQOOP-1988
> Project: Sqoop
> Issue Type: Sub-task
> Reporter: Qian Xu
> Assignee: Qian Xu
> Fix For: 2.0.0
>
>
> Matcher should only match fields, not do any value modification (except when fallback to null ), Given argument {{fields}} should have converted null values.
> Currently it will convert {{null}} {{"NULL"}} {{"null"}} {{"'null'"}} and {{""}} to null. There are two questions here:
> 1. They are CSV relevant. As we do not have a knowledge of what intermediate data format is used, why we do the check and convert here.
> 2. {{"'null'"}} (double quote and single quote), it is a valid string represents single quote null single quote. It will be converted to null. Is this valid?
> I'd propose to remove it. And it will have performance advance. Thanks [~jerrychenhf]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)