You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by "Kishore Veleti (JIRA)" <ji...@apache.org> on 2015/07/18 03:18:04 UTC
[jira] [Commented] (SQOOP-2271) HCatalog ORC Null Pointer Exception
[ https://issues.apache.org/jira/browse/SQOOP-2271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14632193#comment-14632193 ]
Kishore Veleti commented on SQOOP-2271:
---------------------------------------
Hi Hari,
Did you tried by specifying --hcatalog-database "<database name>" option?
I tried similar query on Mysql and even I got the error. Later I added --hcatalog-database and sqoop import executed successfully.
If you can execute in your environment " hcat -f,/tmp/hcat-script-1427459532585 " you will get an error "FAILED: SemanticException org.apache.hadoop.hive.ql.parse.SemanticException: Invalid table name default."
(Note -f <file name> is shown in your initial message)
Below query worked without any issue - ( i added hcatalog-database option and also removed the HCatalog database name from the --hcatalog-table option
-------------------------------------------------------
sqoop import --connect "jdbc:mysql://127.0.0.1/metastore" --username "etluser" --password "etluser" --verbose -m 1 --query 'select * from metastore.TBLS WHERE 1=1 AND $CONDITIONS' --mapreduce-job-name "sqoop-import-myJob" --create-hcatalog-table --hcatalog-database "sqoop146_db" --hcatalog-table "metastore_tbls" --hcatalog-storage-stanza "STORED AS ORC"
Below query failed because I did not specified --hcatalog-database option but specified the database name along with table name
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
sqoop import --connect "jdbc:mysql://127.0.0.1/metastore" --username "etluser" --password "etluser" --verbose -m 1 --query 'select * from metastore.TBLS WHERE 1=1 AND $CONDITIONS' --mapreduce-job-name "sqoop-import-myJob" --create-hcatalog-table --hcatalog-table "sqoop146_db.metastore_tbls" --hcatalog-storage-stanza "STORED AS ORC"
15/07/17 18:14:08 INFO hcat.SqoopHCatUtilities: Executing external HCatalog CLI process with args :-f,/tmp/hcat-script-1437182048694
15/07/17 18:14:22 INFO hcat.SqoopHCatUtilities: FAILED: SemanticException org.apache.hadoop.hive.ql.parse.SemanticException: Invalid table name default.sqoop146_db.metastore_tbls
15/07/17 18:14:22 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: HCat exited with status 64
at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.executeExternalHCatProgram(SqoopHCatUtilities.java:1142)
at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.launchHCatCli(SqoopHCatUtilities.java:1091)
at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.createHCatTable(SqoopHCatUtilities.java:624)
at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:339)
at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureImportOutputFormat(SqoopHCatUtilities.java:782)
at org.apache.sqoop.mapreduce.ImportJobBase.configureOutputFormat(ImportJobBase.java:98)
at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:259)
at org.apache.sqoop.manager.SqlManager.importQuery(SqlManager.java:724)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:499)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
> HCatalog ORC Null Pointer Exception
> -----------------------------------
>
> Key: SQOOP-2271
> URL: https://issues.apache.org/jira/browse/SQOOP-2271
> Project: Sqoop
> Issue Type: Bug
> Components: codegen, hive-integration
> Affects Versions: 1.4.5
> Environment: HDP 2.2
> Reporter: Hari Sekhon
> Priority: Blocker
>
> Hitting a Null Pointer Exception when importing a table via HCatalog to ORC. We do a lot of SQL Server imports, the only additions are the hcatalog / orc options:{code}sqoop import --connect "jdbc:sqlserver://host:1533;database=myDB" --username "myUser" --password "myPassword" --verbose -m 1 --query "select * from [blah].[dbo].[blah2]" --mapreduce-job-name "sqoop-import-myJob" --create-hcatalog-table --hcatalog-table "mydb.mytable" --hcatalog-storage-stanza "STORED AS ORC"
> 15/03/27 12:32:12 INFO hcat.SqoopHCatUtilities: Executing external HCatalog CLI process with args :-f,/tmp/hcat-script-1427459532585
> 15/03/27 12:32:12 DEBUG util.ClassLoaderStack: Restoring classloader: java.net.URLClassLoader@7640f42a
> 15/03/27 12:32:12 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.NullPointerException
> java.lang.NullPointerException
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:1010)
> at java.lang.Runtime.exec(Runtime.java:617)
> at java.lang.Runtime.exec(Runtime.java:528)
> at org.apache.sqoop.util.Executor.exec(Executor.java:76)
> at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.executeExternalHCatProgram(SqoopHCatUtilities.java:1119)
> at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.launchHCatCli(SqoopHCatUtilities.java:1071)
> at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.createHCatTable(SqoopHCatUtilities.java:610)
> at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:325)
> at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureImportOutputFormat(SqoopHCatUtilities.java:768)
> at org.apache.sqoop.mapreduce.ImportJobBase.configureOutputFormat(ImportJobBase.java:98)
> at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:249)
> at org.apache.sqoop.manager.SqlManager.importQuery(SqlManager.java:721)
> at org.apache.sqoop.manager.SQLServerManager.importQuery(SQLServerManager.java:346)
> at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:499)
> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:601)
> at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
> {code}
> At first I thought this was SQOOP-1533 but the stack trace is different and we're on Oracle Java 7.
> Hari Sekhon
> http://www.linkedin.com/in/harisekhon
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)