You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@sqoop.apache.org by Kartik Rustagi <ka...@gmail.com> on 2013/06/20 14:31:55 UTC

Dealing with Postgresql schema when doing Hive import using Sqoop1.4.3

Hi,

I am having an issue dealing with the DB->Schema->Table hierarchy of
Postgresql with --hive-import option in sqoop. I am able to successfully
import data to HDFS by mentioning --table as schema_name.table_name and
--split-by as schema_name.table_name.id (I have mentioned the DB in the
--connect option), But when doing --hive-import I get the error:

13/06/20 17:59:56 *INFO hive.HiveImport: FAILED: Error in metadata:
InvalidObjectException(message:There is no database named schema_namel)*
13/06/20 17:59:56 INFO hive.HiveImport: FAILED: Execution Error, return
code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
13/06/20 17:59:56 ERROR tool.ImportTool: Encountered IOException running
import job: java.io.IOException: Hive exited with status 1
    at
org.apache.sqoop.hive.HiveImport.executeExternalHiveScript(HiveImport.java:364)
    at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:314)
    at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:226)
    at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:415)
    at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476)
    at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
    at org.apache.sqoop.Sqoop.main(Sqoop.java:238)




-- 
Regards
Kartik Rustagi

Re: Dealing with Postgresql schema when doing Hive import using Sqoop1.4.3

Posted by Jarek Jarcec Cecho <ja...@apache.org>.
Hi Kartik,
the Sqoop parameter --table should be used only for table name and not for specifying schema. You should use PostgreSQL connector extra argument --schema to enter your custom schema, for example:

  sqoop import --connect ... --table table_name -- --schema schema_name

Jarcec

On Thu, Jun 20, 2013 at 06:01:55PM +0530, Kartik Rustagi wrote:
> Hi,
> 
> I am having an issue dealing with the DB->Schema->Table hierarchy of
> Postgresql with --hive-import option in sqoop. I am able to successfully
> import data to HDFS by mentioning --table as schema_name.table_name and
> --split-by as schema_name.table_name.id (I have mentioned the DB in the
> --connect option), But when doing --hive-import I get the error:
> 
> 13/06/20 17:59:56 *INFO hive.HiveImport: FAILED: Error in metadata:
> InvalidObjectException(message:There is no database named schema_namel)*
> 13/06/20 17:59:56 INFO hive.HiveImport: FAILED: Execution Error, return
> code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
> 13/06/20 17:59:56 ERROR tool.ImportTool: Encountered IOException running
> import job: java.io.IOException: Hive exited with status 1
>     at
> org.apache.sqoop.hive.HiveImport.executeExternalHiveScript(HiveImport.java:364)
>     at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:314)
>     at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:226)
>     at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:415)
>     at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476)
>     at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>     at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
>     at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
> 
> 
> 
> 
> -- 
> Regards
> Kartik Rustagi