You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by "Shigeki Hirose (JIRA)" <ji...@apache.org> on 2014/09/19 06:55:33 UTC
[jira] [Created] (SQOOP-1533) sqoop import --create-hcatalog-table
for mysql causes NullPointerException when column name contains upper case
char
Shigeki Hirose created SQOOP-1533:
-------------------------------------
Summary: sqoop import --create-hcatalog-table for mysql causes NullPointerException when column name contains upper case char
Key: SQOOP-1533
URL: https://issues.apache.org/jira/browse/SQOOP-1533
Project: Sqoop
Issue Type: Bug
Environment: ubuntu, Sqoop 1.4.4-cdh5.1.2, java 8
Reporter: Shigeki Hirose
Priority: Critical
Sqoop command example:
{noformat}
sqoop import --connect "jdbc:mysql://127.0.0.1:3306/xxx" --username xxx --password xxx --table SampleTable --hcatalog-database xxx --create-hcatalog-table --hcatalog-storage-stanza 'stored as orc' --hcatalog-table sampletable
{noformat}
Call stack:
{noformat}
sqoop.Sqoop: Got exception running Sqoop: java.lang.NullPointerException
java.lang.NullPointerException
at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.createHCatTable(SqoopHCatUtilities.java:491)
at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:293)
at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureImportOutputFormat(SqoopHCatUtilities.java:657)
at org.apache.sqoop.mapreduce.ImportJobBase.configureOutputFormat(ImportJobBase.java:98)
at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:240)
at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:614)
at org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:118)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:413)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:506)
at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231)
at org.apache.sqoop.Sqoop.main(Sqoop.java:240)
{noformat}
This seems to be because source MySQL table contains a column name that has an upper case letter.
Sample table below. The column name, 'A', causes the exception.
{code:sql}
DROP TABLE `SampleTable`;
CREATE TABLE `SampleTable` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`A` bigint(20) unsigned NOT NULL DEFAULT '0',
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
{code}
Once column name is changed to 'a', it works properly:
{code:sql}
DROP TABLE `SampleTable`;
CREATE TABLE `SampleTable` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`a` bigint(20) unsigned NOT NULL DEFAULT '0',
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
{code}
This is probably because LCKeyMap (and HCKeyMap) class in SqoopHCatUtilities.java line 160 - 177 are expecting HashMap to call *put* method for each element when *putAll* is invoked, which seem to have been the case in Java 6/7, but it doesn't behave that way in Java 8.
{code:java}
/**
* A Map using String as key type that ignores case of its key and stores the
* key in lower case.
*/
private static class LCKeyMap<V> extends HashMap<String, V> {
private static final long serialVersionUID = -6751510232323094216L;
@Override
public V put(String key, V value) {
return super.put(key.toLowerCase(), value);
}
@Override
public V get(Object key) {
return super.get(((String) key).toLowerCase());
}
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)