You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by "wanghao (JIRA)" <ji...@apache.org> on 2019/04/04 10:58:00 UTC
[jira] [Created] (KYLIN-3934) sqoop import param '--null-string'
result in null value become blank string in hive table
wanghao created KYLIN-3934:
------------------------------
Summary: sqoop import param '--null-string' result in null value become blank string in hive table
Key: KYLIN-3934
URL: https://issues.apache.org/jira/browse/KYLIN-3934
Project: Kylin
Issue Type: Bug
Components: Others
Affects Versions: v2.6.0
Reporter: wanghao
when column value from jdbc is null, sqoop will write it into hive table with blank string.
eg
jdbc:
A | B
1 | 1
2 | 2
a | null
hive table:
A | B
1 | 1
2 | 2
a |
because of this, when I use count(distinct B), it return 3, not 2, and it can lead to other problems
{code:java}
String cmd = String.format(Locale.ROOT,
"%s/bin/sqoop import" + generateSqoopConfigArgString()
+ "--connect \"%s\" --driver %s --username %s --password %s --query \"%s AND \\$CONDITIONS\" "
+ "--target-dir %s/%s --split-by %s --boundary-query \"%s\" --null-string '' "
+ "--fields-terminated-by '%s' --num-mappers %d",
sqoopHome, connectionUrl, driverClass, jdbcUser, jdbcPass, selectSql, jobWorkingDir, hiveTable,
splitColumn, bquery, filedDelimiter, mapperNum);
{code}
the param '–null=string' should be '\\N' instead of blank string ''.
I resolved this problem by replace the param. But it needs be configured in kylin.properties
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)