You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@sqoop.apache.org by "Kallinteris, Christos" <ka...@initions.com> on 2013/10/30 14:36:35 UTC
Sqoop 1 Importing tables from non DBO schema
Hello!
I'm trying to import a Table into Hive with Sqoop 1.4.3 that has a nonstandard schema. If I just reference the table I get an invalid object name exception.
The documentation for Sqoop says that when importing from SQLServer one can specify the schema with the -schema option but when I try that I get an unrecognized parameter error.
Import of dbo Tables works so it's not the rest of the connection string that's broken. I can import the tables using Sqoop2 but then I don't have the -hive option :(
Any ideas and recommendations welcome.
Thanks!
Christos
--
initions AG
Christos Kallinteris
Weidestraße 120a
D-22083 Hamburg
t: +49 (0) 40 / 41 49 60-79
f: +49 (0) 40 / 41 49 60-11
e: kallinteris@initions.com<ma...@initions.com>
w: www.initions.com<http://www.initions.com/>
Vollständiger Name der Gesellschaft: initions innovative IT solutions AG
Sitz der Gesellschaft: Hamburg
Handelsregister Hamburg B 83929
Aufsichtsratsvorsitzender: Dr. Michael Leue
Vorstand: Dr. Stefan Anschütz, André Paul Henkel, Dr. Helge Plehn
Re: Sqoop 1 Importing tables from non DBO schema
Posted by Joanne Chan <jc...@shutterstock.com>.
I specified the schema / database within the jdbc connection string and it
works fine.
Give that a try.
On Thu, Oct 31, 2013 at 4:47 AM, Kallinteris, Christos <
kallinteris@initions.com> wrote:
> Hello Abe,****
>
> ** **
>
> The command is as follows:****
>
> ** **
>
> sqoop import --connect "jdbc:sqlserver://192.168.207.177\\SQL2012;user=sqoop;password=password;databasename=AdventureWorks2008R2;"
> --driver com.microsoft.sqlserver.jdbc.SQLServerDriver --username sqoop
> --password password --hive-import --table Product --schema Production****
>
> ** **
>
> If I omit the schema parameter it complains about an invalid object name,
> if I include it, it complains about an unrecognized parameter.****
>
> ** **
>
> If this is a limitation of Sqoop, I will import using Sqoop2 and manually
> load the tables into hive; I was just hoping to avoid the hassle of writing
> the table definitions for hive myself.****
>
> ** **
>
> Thanks for your time!****
>
> ** **
>
> Christos****
>
> *--*
>
> * *
>
> *initions AG
> Christos Kallinteris
> Weidestraße 120a
> D-22083 Hamburg
> **
> **t: +49 (0) 40 / 41 49 60-79
> f: +49 (0) 40 / 41 49 60-11
> e: kallinteris@initions.com
> w: www.initions.com
> *
>
> * *
>
> *Vollständiger Name der Gesellschaft: initions innovative IT solutions AG*
>
> *Sitz der Gesellschaft: Hamburg
> Handelsregister Hamburg B 83929
> Aufsichtsratsvorsitzender: **Dr. Michael Leue**
> Vorstand: Dr. Stefan Anschütz, André Paul Henkel, Dr. Helge Plehn*
>
> ** **
>
> *Von:* Abraham Elmahrek [mailto:abe@cloudera.com]
> *Gesendet:* Mittwoch, 30. Oktober 2013 19:26
> *An:* user@sqoop.apache.org
> *Betreff:* Re: Sqoop 1 Importing tables from non DBO schema****
>
> ** **
>
> Hey there,****
>
> ** **
>
> Can you provide the sqoop 1.4.3 command you're using?****
>
> ** **
>
> If you can import via sqoop2, but are missing the proper commands to
> import into hive, you may be able to simply load the data into hive once
> it's in HDFS (
> https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-DMLOperations
> ).****
>
> ** **
>
> -Abe****
>
> ** **
>
> On Wed, Oct 30, 2013 at 6:36 AM, Kallinteris, Christos <
> kallinteris@initions.com> wrote:****
>
> Hello!****
>
> ****
>
> I’m trying to import a Table into Hive with Sqoop 1.4.3 that has a
> nonstandard schema. If I just reference the table I get an invalid object
> name exception. ****
>
> ****
>
> The documentation for Sqoop says that when importing from SQLServer one
> can specify the schema with the –schema option but when I try that I get an
> unrecognized parameter error.****
>
> ****
>
> Import of dbo Tables works so it’s not the rest of the connection string
> that’s broken. I can import the tables using Sqoop2 but then I don’t have
> the –hive option L****
>
> ****
>
> Any ideas and recommendations welcome.****
>
> ****
>
> Thanks!****
>
> ****
>
> Christos****
>
> ****
>
> *--*****
>
> * *****
>
> *initions AG
> Christos Kallinteris
> Weidestraße 120a
> D-22083 Hamburg
> **
> t: +49 (0) 40 / 41 49 60-79
> f: +49 (0) 40 / 41 49 60-11
> e: kallinteris@initions.com
> w: www.initions.com
> *****
>
> * *****
>
> *Vollständiger Name der Gesellschaft: initions innovative IT solutions AG*
> ****
>
> *Sitz der Gesellschaft: Hamburg
> Handelsregister Hamburg B 83929
> Aufsichtsratsvorsitzender: Dr. Michael Leue
> Vorstand: Dr. Stefan Anschütz, André Paul Henkel, Dr. Helge Plehn*****
>
> ****
>
> ** **
>
--
-- JChan
AW: Sqoop 1 Importing tables from non DBO schema
Posted by "Kallinteris, Christos" <ka...@initions.com>.
Hello Jarcec,
I tried that, it doesn't work.
Embarrassingly what does work is simply prepending the Schema to the Table name ( Schema.Table). I could have sworn, I had tried that already, but that works.
Just for information: When you do upload Tables from different schemas into Hive you have to create databases for each schema as Hive translates schema into database and will complain, if you haven't created them.
Thanks for your time!
Christos
--
initions AG
Christos Kallinteris
Weidestraße 120a
D-22083 Hamburg
t: +49 (0) 40 / 41 49 60-79
f: +49 (0) 40 / 41 49 60-11
e: kallinteris@initions.com
w: www.initions.com
Vollständiger Name der Gesellschaft: initions innovative IT solutions AG
Sitz der Gesellschaft: Hamburg
Handelsregister Hamburg B 83929
Aufsichtsratsvorsitzender: Dr. Michael Leue
Vorstand: Dr. Stefan Anschütz, André Paul Henkel, Dr. Helge Plehn
-----Ursprüngliche Nachricht-----
Von: Jarek Jarcec Cecho [mailto:jarcec@apache.org]
Gesendet: Donnerstag, 31. Oktober 2013 16:30
An: user@sqoop.apache.org
Betreff: Re: Sqoop 1 Importing tables from non DBO schema
Hi Christos,
parameter --schema is a connector specific "extra" parameter, not a "sqoop" parameter and therefore you have to place it into the section for extra arguments, for example:
sqoop import --connect .... -- --schema Production
Jarcec
On Thu, Oct 31, 2013 at 08:47:00AM +0000, Kallinteris, Christos wrote:
> Hello Abe,
>
> The command is as follows:
>
> sqoop import --connect
> "jdbc:sqlserver://192.168.207.177\\SQL2012;user=sqoop;password=passwor
> d;databasename=AdventureWorks2008R2;" --driver
> com.microsoft.sqlserver.jdbc.SQLServerDriver --username sqoop
> --password password --hive-import --table Product --schema Production
>
> If I omit the schema parameter it complains about an invalid object name, if I include it, it complains about an unrecognized parameter.
>
> If this is a limitation of Sqoop, I will import using Sqoop2 and manually load the tables into hive; I was just hoping to avoid the hassle of writing the table definitions for hive myself.
>
> Thanks for your time!
>
> Christos
> --
>
> initions AG
> Christos Kallinteris
> Weidestraße 120a
> D-22083 Hamburg
>
> t: +49 (0) 40 / 41 49 60-79
> f: +49 (0) 40 / 41 49 60-11
> e: kallinteris@initions.com<ma...@initions.com>
> w: www.initions.com<http://www.initions.com/>
>
>
> Vollständiger Name der Gesellschaft: initions innovative IT solutions
> AG Sitz der Gesellschaft: Hamburg Handelsregister Hamburg B 83929
> Aufsichtsratsvorsitzender: Dr. Michael Leue
> Vorstand: Dr. Stefan Anschütz, André Paul Henkel, Dr. Helge Plehn
>
> Von: Abraham Elmahrek [mailto:abe@cloudera.com]
> Gesendet: Mittwoch, 30. Oktober 2013 19:26
> An: user@sqoop.apache.org
> Betreff: Re: Sqoop 1 Importing tables from non DBO schema
>
> Hey there,
>
> Can you provide the sqoop 1.4.3 command you're using?
>
> If you can import via sqoop2, but are missing the proper commands to import into hive, you may be able to simply load the data into hive once it's in HDFS (https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-DMLOperations).
>
> -Abe
>
> On Wed, Oct 30, 2013 at 6:36 AM, Kallinteris, Christos <ka...@initions.com>> wrote:
> Hello!
>
> I'm trying to import a Table into Hive with Sqoop 1.4.3 that has a nonstandard schema. If I just reference the table I get an invalid object name exception.
>
> The documentation for Sqoop says that when importing from SQLServer one can specify the schema with the -schema option but when I try that I get an unrecognized parameter error.
>
> Import of dbo Tables works so it's not the rest of the connection
> string that's broken. I can import the tables using Sqoop2 but then I
> don't have the -hive option :(
>
> Any ideas and recommendations welcome.
>
> Thanks!
>
> Christos
>
> --
>
> initions AG
> Christos Kallinteris
> Weidestraße 120a
> D-22083 Hamburg
>
> t: +49 (0) 40 / 41 49 60-79<tel:%2B49%20%280%29%2040%20%2F%2041%2049%2060-79>
> f: +49 (0) 40 / 41 49 60-11<tel:%2B49%20%280%29%2040%20%2F%2041%2049%2060-11>
> e: kallinteris@initions.com<ma...@initions.com>
> w: www.initions.com<http://www.initions.com/>
>
>
> Vollständiger Name der Gesellschaft: initions innovative IT solutions
> AG Sitz der Gesellschaft: Hamburg Handelsregister Hamburg B 83929
> Aufsichtsratsvorsitzender: Dr. Michael Leue
> Vorstand: Dr. Stefan Anschütz, André Paul Henkel, Dr. Helge Plehn
>
>
Re: Sqoop 1 Importing tables from non DBO schema
Posted by Jarek Jarcec Cecho <ja...@apache.org>.
Hi Christos,
parameter --schema is a connector specific "extra" parameter, not a "sqoop" parameter and therefore you have to place it into the section for extra arguments, for example:
sqoop import --connect .... -- --schema Production
Jarcec
On Thu, Oct 31, 2013 at 08:47:00AM +0000, Kallinteris, Christos wrote:
> Hello Abe,
>
> The command is as follows:
>
> sqoop import --connect "jdbc:sqlserver://192.168.207.177\\SQL2012;user=sqoop;password=password;databasename=AdventureWorks2008R2;" --driver com.microsoft.sqlserver.jdbc.SQLServerDriver --username sqoop --password password --hive-import --table Product --schema Production
>
> If I omit the schema parameter it complains about an invalid object name, if I include it, it complains about an unrecognized parameter.
>
> If this is a limitation of Sqoop, I will import using Sqoop2 and manually load the tables into hive; I was just hoping to avoid the hassle of writing the table definitions for hive myself.
>
> Thanks for your time!
>
> Christos
> --
>
> initions AG
> Christos Kallinteris
> Weidestraße 120a
> D-22083 Hamburg
>
> t: +49 (0) 40 / 41 49 60-79
> f: +49 (0) 40 / 41 49 60-11
> e: kallinteris@initions.com<ma...@initions.com>
> w: www.initions.com<http://www.initions.com/>
>
>
> Vollständiger Name der Gesellschaft: initions innovative IT solutions AG
> Sitz der Gesellschaft: Hamburg
> Handelsregister Hamburg B 83929
> Aufsichtsratsvorsitzender: Dr. Michael Leue
> Vorstand: Dr. Stefan Anschütz, André Paul Henkel, Dr. Helge Plehn
>
> Von: Abraham Elmahrek [mailto:abe@cloudera.com]
> Gesendet: Mittwoch, 30. Oktober 2013 19:26
> An: user@sqoop.apache.org
> Betreff: Re: Sqoop 1 Importing tables from non DBO schema
>
> Hey there,
>
> Can you provide the sqoop 1.4.3 command you're using?
>
> If you can import via sqoop2, but are missing the proper commands to import into hive, you may be able to simply load the data into hive once it's in HDFS (https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-DMLOperations).
>
> -Abe
>
> On Wed, Oct 30, 2013 at 6:36 AM, Kallinteris, Christos <ka...@initions.com>> wrote:
> Hello!
>
> I'm trying to import a Table into Hive with Sqoop 1.4.3 that has a nonstandard schema. If I just reference the table I get an invalid object name exception.
>
> The documentation for Sqoop says that when importing from SQLServer one can specify the schema with the -schema option but when I try that I get an unrecognized parameter error.
>
> Import of dbo Tables works so it's not the rest of the connection string that's broken. I can import the tables using Sqoop2 but then I don't have the -hive option :(
>
> Any ideas and recommendations welcome.
>
> Thanks!
>
> Christos
>
> --
>
> initions AG
> Christos Kallinteris
> Weidestraße 120a
> D-22083 Hamburg
>
> t: +49 (0) 40 / 41 49 60-79<tel:%2B49%20%280%29%2040%20%2F%2041%2049%2060-79>
> f: +49 (0) 40 / 41 49 60-11<tel:%2B49%20%280%29%2040%20%2F%2041%2049%2060-11>
> e: kallinteris@initions.com<ma...@initions.com>
> w: www.initions.com<http://www.initions.com/>
>
>
> Vollständiger Name der Gesellschaft: initions innovative IT solutions AG
> Sitz der Gesellschaft: Hamburg
> Handelsregister Hamburg B 83929
> Aufsichtsratsvorsitzender: Dr. Michael Leue
> Vorstand: Dr. Stefan Anschütz, André Paul Henkel, Dr. Helge Plehn
>
>
AW: Sqoop 1 Importing tables from non DBO schema
Posted by "Kallinteris, Christos" <ka...@initions.com>.
Hello Abe,
The command is as follows:
sqoop import --connect "jdbc:sqlserver://192.168.207.177\\SQL2012;user=sqoop;password=password;databasename=AdventureWorks2008R2;" --driver com.microsoft.sqlserver.jdbc.SQLServerDriver --username sqoop --password password --hive-import --table Product --schema Production
If I omit the schema parameter it complains about an invalid object name, if I include it, it complains about an unrecognized parameter.
If this is a limitation of Sqoop, I will import using Sqoop2 and manually load the tables into hive; I was just hoping to avoid the hassle of writing the table definitions for hive myself.
Thanks for your time!
Christos
--
initions AG
Christos Kallinteris
Weidestraße 120a
D-22083 Hamburg
t: +49 (0) 40 / 41 49 60-79
f: +49 (0) 40 / 41 49 60-11
e: kallinteris@initions.com<ma...@initions.com>
w: www.initions.com<http://www.initions.com/>
Vollständiger Name der Gesellschaft: initions innovative IT solutions AG
Sitz der Gesellschaft: Hamburg
Handelsregister Hamburg B 83929
Aufsichtsratsvorsitzender: Dr. Michael Leue
Vorstand: Dr. Stefan Anschütz, André Paul Henkel, Dr. Helge Plehn
Von: Abraham Elmahrek [mailto:abe@cloudera.com]
Gesendet: Mittwoch, 30. Oktober 2013 19:26
An: user@sqoop.apache.org
Betreff: Re: Sqoop 1 Importing tables from non DBO schema
Hey there,
Can you provide the sqoop 1.4.3 command you're using?
If you can import via sqoop2, but are missing the proper commands to import into hive, you may be able to simply load the data into hive once it's in HDFS (https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-DMLOperations).
-Abe
On Wed, Oct 30, 2013 at 6:36 AM, Kallinteris, Christos <ka...@initions.com>> wrote:
Hello!
I'm trying to import a Table into Hive with Sqoop 1.4.3 that has a nonstandard schema. If I just reference the table I get an invalid object name exception.
The documentation for Sqoop says that when importing from SQLServer one can specify the schema with the -schema option but when I try that I get an unrecognized parameter error.
Import of dbo Tables works so it's not the rest of the connection string that's broken. I can import the tables using Sqoop2 but then I don't have the -hive option :(
Any ideas and recommendations welcome.
Thanks!
Christos
--
initions AG
Christos Kallinteris
Weidestraße 120a
D-22083 Hamburg
t: +49 (0) 40 / 41 49 60-79<tel:%2B49%20%280%29%2040%20%2F%2041%2049%2060-79>
f: +49 (0) 40 / 41 49 60-11<tel:%2B49%20%280%29%2040%20%2F%2041%2049%2060-11>
e: kallinteris@initions.com<ma...@initions.com>
w: www.initions.com<http://www.initions.com/>
Vollständiger Name der Gesellschaft: initions innovative IT solutions AG
Sitz der Gesellschaft: Hamburg
Handelsregister Hamburg B 83929
Aufsichtsratsvorsitzender: Dr. Michael Leue
Vorstand: Dr. Stefan Anschütz, André Paul Henkel, Dr. Helge Plehn
Re: Sqoop 1 Importing tables from non DBO schema
Posted by Abraham Elmahrek <ab...@cloudera.com>.
Hey there,
Can you provide the sqoop 1.4.3 command you're using?
If you can import via sqoop2, but are missing the proper commands to import
into hive, you may be able to simply load the data into hive once it's in
HDFS (
https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-DMLOperations
).
-Abe
On Wed, Oct 30, 2013 at 6:36 AM, Kallinteris, Christos <
kallinteris@initions.com> wrote:
> Hello!****
>
> ** **
>
> I’m trying to import a Table into Hive with Sqoop 1.4.3 that has a
> nonstandard schema. If I just reference the table I get an invalid object
> name exception. ****
>
> ** **
>
> The documentation for Sqoop says that when importing from SQLServer one
> can specify the schema with the –schema option but when I try that I get an
> unrecognized parameter error.****
>
> ** **
>
> Import of dbo Tables works so it’s not the rest of the connection string
> that’s broken. I can import the tables using Sqoop2 but then I don’t have
> the –hive option L****
>
> ** **
>
> Any ideas and recommendations welcome.****
>
> ** **
>
> Thanks!****
>
> ** **
>
> Christos****
>
> ** **
>
> *--*
>
> * *
>
> *initions AG
> Christos Kallinteris
> Weidestraße 120a
> D-22083 Hamburg
> **
> **t: +49 (0) 40 / 41 49 60-79
> f: +49 (0) 40 / 41 49 60-11
> e: kallinteris@initions.com
> w: www.initions.com
> *
>
> * *
>
> *Vollständiger Name der Gesellschaft: initions innovative IT solutions AG*
>
> *Sitz der Gesellschaft: Hamburg
> Handelsregister Hamburg B 83929
> Aufsichtsratsvorsitzender: **Dr. Michael Leue**
> Vorstand: Dr. Stefan Anschütz, André Paul Henkel, Dr. Helge Plehn*
>
> ** **
>