You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@sqoop.apache.org by Brian Jeltema <br...@digitalenvoy.net> on 2015/03/26 16:48:56 UTC
sqoop failure when validating fields
I’ve recently installed a new development cluster (HDP 2.1)
The sqoop jar is sqoop-1.4.4.2.1.7.0-784.jar
I’m trying to import a table directly into HCat using a query like:
sqoop import\
--connect jdbc:oracle:thin:@oracle-host:1521:testdb\
--username foo\
--password bar\
-m 6\
--query "select city_key,
state_key,
country,
state_short region,
city_name,
metro_name,
'foo' data_status
from city_metro
where \$CONDITIONS" --split-by city_key --hcatalog-database reference --hcatalog-table cm
where the target table exists and column names are correct. This import works correctly on an older production cluster, but
in the new cluster I keep getting the exception below. The following two debug lines are not displayed when this
is run on the old cluster:
2015-03-26 11:25:37,255 [main] DEBUG org.apache.sqoop.util.ClassLoaderStack - Restoring classloader: sun.misc.Launcher$AppClassLoader@12843fce
2015-03-26 11:25:37,255 [main] DEBUG org.apache.sqoop.manager.OracleManager$ConnCache - Caching released connection for jdbc:oracle:thin:@oracle-dev:1521:acuity/whipp
Does this failure make sense to anyone?
Thanks
Brian
2015-03-26 11:25:37,254 [main] INFO org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities - Validating dynamic partition keys
2015-03-26 11:25:37,255 [main] DEBUG org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities - No user defined type mapping for HCatalog field city_key
2015-03-26 11:25:37,255 [main] DEBUG org.apache.sqoop.util.ClassLoaderStack - Restoring classloader: sun.misc.Launcher$AppClassLoader@12843fce
2015-03-26 11:25:37,255 [main] DEBUG org.apache.sqoop.manager.OracleManager$ConnCache - Caching released connection for jdbc:oracle:thin:@oracle-dev:1521:acuity/whipp
2015-03-26 11:25:37,255 [main] ERROR org.apache.sqoop.Sqoop - Got exception running Sqoop: java.lang.NullPointerException
java.lang.NullPointerException
at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.validateFieldAndColumnMappings(SqoopHCatUtilities.java:552)
at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureImportOutputFormat(SqoopHCatUtilities.java:660)
at org.apache.sqoop.mapreduce.ImportJobBase.configureOutputFormat(ImportJobBase.java:98)
at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:240)
at org.apache.sqoop.manager.SqlManager.importQuery(SqlManager.java:670)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:415)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:506)
at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231)
at org.apache.sqoop.Sqoop.main(Sqoop.java:240)
Re: sqoop failure when validating fields
Posted by billy iftissen <bi...@gmail.com>.
Try to use:
-connect jdbc:oracle:thin:@oracle-host:1521/testdb
not
-connect jdbc:oracle:thin:@oracle-host:1521:testdb
Hope this helps you.
Le 26 mars 2015 16:49, "Brian Jeltema" <br...@digitalenvoy.net> a
écrit :
> I’ve recently installed a new development cluster (HDP 2.1)
>
> The sqoop jar is sqoop-1.4.4.2.1.7.0-784.jar
>
> I’m trying to import a table directly into HCat using a query like:
>
> sqoop import\
> --connect jdbc:oracle:thin:@oracle-host:1521:testdb\
> --username foo\
> --password bar\
> -m 6\
> --query "select city_key,
> state_key,
> country,
> state_short region,
> city_name,
> metro_name,
> 'foo' data_status
> from city_metro
> where \$CONDITIONS" --split-by city_key
> --hcatalog-database reference --hcatalog-table cm
>
> where the target table exists and column names are correct. This import
> works correctly on an older production cluster, but
> in the new cluster I keep getting the exception below. The following two
> debug lines are not displayed when this
> is run on the old cluster:
>
> 2015-03-26 11:25:37,255 [main] DEBUG
> org.apache.sqoop.util.ClassLoaderStack - Restoring classloader:
> sun.misc.Launcher$AppClassLoader@12843fce
> 2015-03-26 11:25:37,255 [main] DEBUG
> org.apache.sqoop.manager.OracleManager$ConnCache - Caching released
> connection for jdbc:oracle:thin:@oracle-dev:1521:acuity/whipp
>
> Does this failure make sense to anyone?
>
> Thanks
> Brian
>
>
> 2015-03-26 11:25:37,254 [main] INFO
> org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities - Validating dynamic
> partition keys
> 2015-03-26 11:25:37,255 [main] DEBUG
> org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities - No user defined type
> mapping for HCatalog field city_key
> 2015-03-26 11:25:37,255 [main] DEBUG
> org.apache.sqoop.util.ClassLoaderStack - Restoring classloader:
> sun.misc.Launcher$AppClassLoader@12843fce
> 2015-03-26 11:25:37,255 [main] DEBUG
> org.apache.sqoop.manager.OracleManager$ConnCache - Caching released
> connection for jdbc:oracle:thin:@oracle-dev:1521:acuity/whipp
> 2015-03-26 11:25:37,255 [main] ERROR org.apache.sqoop.Sqoop - Got
> exception running Sqoop: java.lang.NullPointerException
> java.lang.NullPointerException
> at
> org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.validateFieldAndColumnMappings(SqoopHCatUtilities.java:552)
> at
> org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureImportOutputFormat(SqoopHCatUtilities.java:660)
> at
> org.apache.sqoop.mapreduce.ImportJobBase.configureOutputFormat(ImportJobBase.java:98)
> at
> org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:240)
> at org.apache.sqoop.manager.SqlManager.importQuery(SqlManager.java:670)
> at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:415)
> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:506)
> at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:240)
>
>