You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@sqoop.apache.org by Lee S <sl...@gmail.com> on 2015/07/07 11:59:24 UTC
Does sqoop2 surpport import from database to hive?
Hi all:
Does hive jdbc can be used as a jdbc connector which can support
importing data to hive?
Re: Does sqoop2 surpport import from database to hive?
Posted by Lee S <sl...@gmail.com>.
Got it, the released 1.99.6 is RC now.
On Mon, Jul 13, 2015 at 11:19 AM, Zhou, Richard <ri...@intel.com>
wrote:
> Actually, there is a mis-understanding.
>
> The latest version of Sqoop 2 is 1.99.6, which has already released. But
> this release (1.99.6) is not GA or production deployment. Some features
> (etc. import/export to hbase, kafka) are still under developing. The GA
> version (maybe 2.0) is target on the end of 2015 or early 2016. But the
> feature of “import into hive via kite connector” is ready now in 1.99.6. So
> you could use this feature now.
>
>
>
>
>
> Regards
>
> Richard
>
>
>
> *From:* Lee S [mailto:sleefd@gmail.com]
> *Sent:* Monday, July 13, 2015 9:48 AM
>
> *To:* user@sqoop.apache.org
> *Subject:* Re: Does sqoop2 surpport import from database to hive?
>
>
>
> http://sqoop.apache.org/
>
>
>
> The first paragraph.
>
>
>
> On Mon, Jul 13, 2015 at 9:45 AM, Zhou, Richard <ri...@intel.com>
> wrote:
>
> May I have the website? We will update it. Thanks.
>
>
>
>
>
> Regards
>
> Richard
>
>
>
> *From:* Lee S [mailto:sleefd@gmail.com]
> *Sent:* Monday, July 13, 2015 9:43 AM
>
>
> *To:* user@sqoop.apache.org
> *Subject:* Re: Does sqoop2 surpport import from database to hive?
>
>
>
> Many thanks to the detail usage. I will try it.
>
> BTW, official website says that 1.99.6 is not feature complete.
>
>
>
> On Wed, Jul 8, 2015 at 4:37 PM, Zhou, Richard <ri...@intel.com>
> wrote:
>
> I have tested the import into hive via Kite connector as Parquet file.
> Here are some hints.
>
> 1. hive-*.jar (hive-core.jar, hive-exec.jar etc.) are needed. You
> could config the common.loader parameter in file
> server/conf/catalina.properties, or copy jars directly into folder
> server/webapps/sqoop/WEB-INF/lib/.
>
> 2. libfb303-0.9.0.jar, datanucleus-api-jdo-3.2.6.jar,
> datanucleus-core-3.2.10.jar, datanucleus-rdbms-3.2.9.jar are also needed.
>
> 3. Kite link information:
>
> sqoop:000> show link -lid 4
>
> 1 link(s) to show:
>
> link with id 4 and name kite-hive (Enabled: true, Created by root at
> 7/8/15 11:21 AM, Updated by root at 7/8/15 12:45 PM)
>
> Using Connector kite-connector with id 2
>
> Link Configuration
>
> HDFS host and port: server-654:9083
>
> 4. Job information:
>
> sqoop:000> show job -jid 6
>
> 1 job(s) to show:
>
> Job with id 6 and name mysql->kite-hive (Enabled: true, Created by root at
> 7/8/15 11:22 AM, Updated by root at 7/8/15 3:31 PM)
>
> Throttling resources
>
> Extractors:
>
> Loaders:
>
> From link: 2
>
> From database configuration
>
> Schema name:
>
> Table name: test
>
> Table SQL statement:
>
> Table column names:
>
> Partition column name: id
>
> Null value allowed for the partition column:
>
> Boundary query:
>
> Incremental read
>
> Check column:
>
> Last value:
>
> To link: 4
>
> To Kite Dataset Configuration
>
> Dataset URI: dataset:hive:tmp/aaa
>
> File format: PARQUET
>
> 5. After imported, use beeline: beeline -u
> 'jdbc:hive2://server-654:10000/tmp' -n root -p 123456
>
> 6. The data is correct
>
> 0: jdbc:hive2://server-654:10000/tmp> show tables;
>
> +-----------+--+
>
> | tab_name |
>
> +-----------+--+
>
> | aaa |
>
> +-----------+--+
>
> 1 row selected (0.182 seconds)
>
>
>
>
>
> Regards
>
> Richard
>
>
>
> *From:* Zhou, Richard [mailto:richard.zhou@intel.com]
> *Sent:* Wednesday, July 08, 2015 10:27 AM
> *To:* user@sqoop.apache.org
> *Subject:* RE: Does sqoop2 surpport import from database to hive?
>
>
>
> It is supported in 1.99.6, which has already released.
>
> You could use Kite connector to import data to hive.
>
>
>
>
>
> Regards
>
> Richard
>
>
>
> *From:* Lee S [mailto:sleefd@gmail.com <sl...@gmail.com>]
> *Sent:* Wednesday, July 08, 2015 10:18 AM
> *To:* user@sqoop.apache.org
> *Subject:* Re: Does sqoop2 surpport import from database to hive?
>
>
>
> It's fixed on version 1.99.6,when will 1.99.6 be released?
>
>
>
> On Wed, Jul 8, 2015 at 6:32 AM, Abraham Elmahrek <ab...@cloudera.com> wrote:
>
> For Sqoop2, check out this Jira:
> https://issues.apache.org/jira/browse/SQOOP-1998.
>
>
>
> If you're using the latest version of Sqoop, it's included.
>
>
>
> -Abe
>
>
>
> On Tue, Jul 7, 2015 at 3:19 AM, rudra tripathy <ru...@yahoo.com> wrote:
>
> Hi Leo,
>
>
>
> There is no meaning to import data to hive as hive understand only HDFS to
> load and one has to load data into hdfs.
>
>
>
>
>
> <font face="arial black">Thanks and Regards,</font>
>
> <font face="arial black">Rudra Prasad Tripathy</font>
>
> <strong>Sr Technical Architect</strong>
>
> <strong></strong>
>
> <strong></strong>
>
>
>
>
>
> On Tuesday, 7 July 2015 3:30 PM, Lee S <sl...@gmail.com> wrote:
>
>
>
> Hi all:
>
> Does hive jdbc can be used as a jdbc connector which can support
> importing data to hive?
>
>
>
>
>
>
>
>
>
>
>
RE: Does sqoop2 surpport import from database to hive?
Posted by "Zhou, Richard" <ri...@intel.com>.
Actually, there is a mis-understanding.
The latest version of Sqoop 2 is 1.99.6, which has already released. But this release (1.99.6) is not GA or production deployment. Some features (etc. import/export to hbase, kafka) are still under developing. The GA version (maybe 2.0) is target on the end of 2015 or early 2016. But the feature of “import into hive via kite connector” is ready now in 1.99.6. So you could use this feature now.
Regards
Richard
From: Lee S [mailto:sleefd@gmail.com]
Sent: Monday, July 13, 2015 9:48 AM
To: user@sqoop.apache.org
Subject: Re: Does sqoop2 surpport import from database to hive?
http://sqoop.apache.org/
The first paragraph.
On Mon, Jul 13, 2015 at 9:45 AM, Zhou, Richard <ri...@intel.com>> wrote:
May I have the website? We will update it. Thanks.
Regards
Richard
From: Lee S [mailto:sleefd@gmail.com<ma...@gmail.com>]
Sent: Monday, July 13, 2015 9:43 AM
To: user@sqoop.apache.org<ma...@sqoop.apache.org>
Subject: Re: Does sqoop2 surpport import from database to hive?
Many thanks to the detail usage. I will try it.
BTW, official website says that 1.99.6 is not feature complete.
On Wed, Jul 8, 2015 at 4:37 PM, Zhou, Richard <ri...@intel.com>> wrote:
I have tested the import into hive via Kite connector as Parquet file. Here are some hints.
1. hive-*.jar (hive-core.jar, hive-exec.jar etc.) are needed. You could config the common.loader parameter in file server/conf/catalina.properties, or copy jars directly into folder server/webapps/sqoop/WEB-INF/lib/.
2. libfb303-0.9.0.jar, datanucleus-api-jdo-3.2.6.jar, datanucleus-core-3.2.10.jar, datanucleus-rdbms-3.2.9.jar are also needed.
3. Kite link information:
sqoop:000> show link -lid 4
1 link(s) to show:
link with id 4 and name kite-hive (Enabled: true, Created by root at 7/8/15 11:21 AM, Updated by root at 7/8/15 12:45 PM)
Using Connector kite-connector with id 2
Link Configuration
HDFS host and port: server-654:9083
4. Job information:
sqoop:000> show job -jid 6
1 job(s) to show:
Job with id 6 and name mysql->kite-hive (Enabled: true, Created by root at 7/8/15 11:22 AM, Updated by root at 7/8/15 3:31 PM)
Throttling resources
Extractors:
Loaders:
From link: 2
From database configuration
Schema name:
Table name: test
Table SQL statement:
Table column names:
Partition column name: id
Null value allowed for the partition column:
Boundary query:
Incremental read
Check column:
Last value:
To link: 4
To Kite Dataset Configuration
Dataset URI: dataset:hive:tmp/aaa
File format: PARQUET
5. After imported, use beeline: beeline -u 'jdbc:hive2://server-654:10000/tmp' -n root -p 123456
6. The data is correct
0: jdbc:hive2://server-654:10000/tmp> show tables;
+-----------+--+
| tab_name |
+-----------+--+
| aaa |
+-----------+--+
1 row selected (0.182 seconds)
Regards
Richard
From: Zhou, Richard [mailto:richard.zhou@intel.com<ma...@intel.com>]
Sent: Wednesday, July 08, 2015 10:27 AM
To: user@sqoop.apache.org<ma...@sqoop.apache.org>
Subject: RE: Does sqoop2 surpport import from database to hive?
It is supported in 1.99.6, which has already released.
You could use Kite connector to import data to hive.
Regards
Richard
From: Lee S [mailto:sleefd@gmail.com]
Sent: Wednesday, July 08, 2015 10:18 AM
To: user@sqoop.apache.org<ma...@sqoop.apache.org>
Subject: Re: Does sqoop2 surpport import from database to hive?
It's fixed on version 1.99.6,when will 1.99.6 be released?
On Wed, Jul 8, 2015 at 6:32 AM, Abraham Elmahrek <ab...@cloudera.com>> wrote:
For Sqoop2, check out this Jira: https://issues.apache.org/jira/browse/SQOOP-1998.
If you're using the latest version of Sqoop, it's included.
-Abe
On Tue, Jul 7, 2015 at 3:19 AM, rudra tripathy <ru...@yahoo.com>> wrote:
Hi Leo,
There is no meaning to import data to hive as hive understand only HDFS to load and one has to load data into hdfs.
<font face="arial black">Thanks and Regards,</font>
<font face="arial black">Rudra Prasad Tripathy</font>
<strong>Sr Technical Architect</strong>
<strong></strong>
<strong></strong>
On Tuesday, 7 July 2015 3:30 PM, Lee S <sl...@gmail.com>> wrote:
Hi all:
Does hive jdbc can be used as a jdbc connector which can support importing data to hive?
Re: Does sqoop2 surpport import from database to hive?
Posted by Lee S <sl...@gmail.com>.
http://sqoop.apache.org/
The first paragraph.
On Mon, Jul 13, 2015 at 9:45 AM, Zhou, Richard <ri...@intel.com>
wrote:
> May I have the website? We will update it. Thanks.
>
>
>
>
>
> Regards
>
> Richard
>
>
>
> *From:* Lee S [mailto:sleefd@gmail.com]
> *Sent:* Monday, July 13, 2015 9:43 AM
>
> *To:* user@sqoop.apache.org
> *Subject:* Re: Does sqoop2 surpport import from database to hive?
>
>
>
> Many thanks to the detail usage. I will try it.
>
> BTW, official website says that 1.99.6 is not feature complete.
>
>
>
> On Wed, Jul 8, 2015 at 4:37 PM, Zhou, Richard <ri...@intel.com>
> wrote:
>
> I have tested the import into hive via Kite connector as Parquet file.
> Here are some hints.
>
> 1. hive-*.jar (hive-core.jar, hive-exec.jar etc.) are needed. You
> could config the common.loader parameter in file
> server/conf/catalina.properties, or copy jars directly into folder
> server/webapps/sqoop/WEB-INF/lib/.
>
> 2. libfb303-0.9.0.jar, datanucleus-api-jdo-3.2.6.jar,
> datanucleus-core-3.2.10.jar, datanucleus-rdbms-3.2.9.jar are also needed.
>
> 3. Kite link information:
>
> sqoop:000> show link -lid 4
>
> 1 link(s) to show:
>
> link with id 4 and name kite-hive (Enabled: true, Created by root at
> 7/8/15 11:21 AM, Updated by root at 7/8/15 12:45 PM)
>
> Using Connector kite-connector with id 2
>
> Link Configuration
>
> HDFS host and port: server-654:9083
>
> 4. Job information:
>
> sqoop:000> show job -jid 6
>
> 1 job(s) to show:
>
> Job with id 6 and name mysql->kite-hive (Enabled: true, Created by root at
> 7/8/15 11:22 AM, Updated by root at 7/8/15 3:31 PM)
>
> Throttling resources
>
> Extractors:
>
> Loaders:
>
> From link: 2
>
> From database configuration
>
> Schema name:
>
> Table name: test
>
> Table SQL statement:
>
> Table column names:
>
> Partition column name: id
>
> Null value allowed for the partition column:
>
> Boundary query:
>
> Incremental read
>
> Check column:
>
> Last value:
>
> To link: 4
>
> To Kite Dataset Configuration
>
> Dataset URI: dataset:hive:tmp/aaa
>
> File format: PARQUET
>
> 5. After imported, use beeline: beeline -u
> 'jdbc:hive2://server-654:10000/tmp' -n root -p 123456
>
> 6. The data is correct
>
> 0: jdbc:hive2://server-654:10000/tmp> show tables;
>
> +-----------+--+
>
> | tab_name |
>
> +-----------+--+
>
> | aaa |
>
> +-----------+--+
>
> 1 row selected (0.182 seconds)
>
>
>
>
>
> Regards
>
> Richard
>
>
>
> *From:* Zhou, Richard [mailto:richard.zhou@intel.com]
> *Sent:* Wednesday, July 08, 2015 10:27 AM
> *To:* user@sqoop.apache.org
> *Subject:* RE: Does sqoop2 surpport import from database to hive?
>
>
>
> It is supported in 1.99.6, which has already released.
>
> You could use Kite connector to import data to hive.
>
>
>
>
>
> Regards
>
> Richard
>
>
>
> *From:* Lee S [mailto:sleefd@gmail.com <sl...@gmail.com>]
> *Sent:* Wednesday, July 08, 2015 10:18 AM
> *To:* user@sqoop.apache.org
> *Subject:* Re: Does sqoop2 surpport import from database to hive?
>
>
>
> It's fixed on version 1.99.6,when will 1.99.6 be released?
>
>
>
> On Wed, Jul 8, 2015 at 6:32 AM, Abraham Elmahrek <ab...@cloudera.com> wrote:
>
> For Sqoop2, check out this Jira:
> https://issues.apache.org/jira/browse/SQOOP-1998.
>
>
>
> If you're using the latest version of Sqoop, it's included.
>
>
>
> -Abe
>
>
>
> On Tue, Jul 7, 2015 at 3:19 AM, rudra tripathy <ru...@yahoo.com> wrote:
>
> Hi Leo,
>
>
>
> There is no meaning to import data to hive as hive understand only HDFS to
> load and one has to load data into hdfs.
>
>
>
>
>
> <font face="arial black">Thanks and Regards,</font>
>
> <font face="arial black">Rudra Prasad Tripathy</font>
>
> <strong>Sr Technical Architect</strong>
>
> <strong></strong>
>
> <strong></strong>
>
>
>
>
>
> On Tuesday, 7 July 2015 3:30 PM, Lee S <sl...@gmail.com> wrote:
>
>
>
> Hi all:
>
> Does hive jdbc can be used as a jdbc connector which can support
> importing data to hive?
>
>
>
>
>
>
>
>
>
RE: Does sqoop2 surpport import from database to hive?
Posted by "Zhou, Richard" <ri...@intel.com>.
May I have the website? We will update it. Thanks.
Regards
Richard
From: Lee S [mailto:sleefd@gmail.com]
Sent: Monday, July 13, 2015 9:43 AM
To: user@sqoop.apache.org
Subject: Re: Does sqoop2 surpport import from database to hive?
Many thanks to the detail usage. I will try it.
BTW, official website says that 1.99.6 is not feature complete.
On Wed, Jul 8, 2015 at 4:37 PM, Zhou, Richard <ri...@intel.com>> wrote:
I have tested the import into hive via Kite connector as Parquet file. Here are some hints.
1. hive-*.jar (hive-core.jar, hive-exec.jar etc.) are needed. You could config the common.loader parameter in file server/conf/catalina.properties, or copy jars directly into folder server/webapps/sqoop/WEB-INF/lib/.
2. libfb303-0.9.0.jar, datanucleus-api-jdo-3.2.6.jar, datanucleus-core-3.2.10.jar, datanucleus-rdbms-3.2.9.jar are also needed.
3. Kite link information:
sqoop:000> show link -lid 4
1 link(s) to show:
link with id 4 and name kite-hive (Enabled: true, Created by root at 7/8/15 11:21 AM, Updated by root at 7/8/15 12:45 PM)
Using Connector kite-connector with id 2
Link Configuration
HDFS host and port: server-654:9083
4. Job information:
sqoop:000> show job -jid 6
1 job(s) to show:
Job with id 6 and name mysql->kite-hive (Enabled: true, Created by root at 7/8/15 11:22 AM, Updated by root at 7/8/15 3:31 PM)
Throttling resources
Extractors:
Loaders:
From link: 2
From database configuration
Schema name:
Table name: test
Table SQL statement:
Table column names:
Partition column name: id
Null value allowed for the partition column:
Boundary query:
Incremental read
Check column:
Last value:
To link: 4
To Kite Dataset Configuration
Dataset URI: dataset:hive:tmp/aaa
File format: PARQUET
5. After imported, use beeline: beeline -u 'jdbc:hive2://server-654:10000/tmp' -n root -p 123456
6. The data is correct
0: jdbc:hive2://server-654:10000/tmp> show tables;
+-----------+--+
| tab_name |
+-----------+--+
| aaa |
+-----------+--+
1 row selected (0.182 seconds)
Regards
Richard
From: Zhou, Richard [mailto:richard.zhou@intel.com<ma...@intel.com>]
Sent: Wednesday, July 08, 2015 10:27 AM
To: user@sqoop.apache.org<ma...@sqoop.apache.org>
Subject: RE: Does sqoop2 surpport import from database to hive?
It is supported in 1.99.6, which has already released.
You could use Kite connector to import data to hive.
Regards
Richard
From: Lee S [mailto:sleefd@gmail.com]
Sent: Wednesday, July 08, 2015 10:18 AM
To: user@sqoop.apache.org<ma...@sqoop.apache.org>
Subject: Re: Does sqoop2 surpport import from database to hive?
It's fixed on version 1.99.6,when will 1.99.6 be released?
On Wed, Jul 8, 2015 at 6:32 AM, Abraham Elmahrek <ab...@cloudera.com>> wrote:
For Sqoop2, check out this Jira: https://issues.apache.org/jira/browse/SQOOP-1998.
If you're using the latest version of Sqoop, it's included.
-Abe
On Tue, Jul 7, 2015 at 3:19 AM, rudra tripathy <ru...@yahoo.com>> wrote:
Hi Leo,
There is no meaning to import data to hive as hive understand only HDFS to load and one has to load data into hdfs.
<font face="arial black">Thanks and Regards,</font>
<font face="arial black">Rudra Prasad Tripathy</font>
<strong>Sr Technical Architect</strong>
<strong></strong>
<strong></strong>
On Tuesday, 7 July 2015 3:30 PM, Lee S <sl...@gmail.com>> wrote:
Hi all:
Does hive jdbc can be used as a jdbc connector which can support importing data to hive?
Re: Does sqoop2 surpport import from database to hive?
Posted by Lee S <sl...@gmail.com>.
Many thanks to the detail usage. I will try it.
BTW, official website says that 1.99.6 is not feature complete.
On Wed, Jul 8, 2015 at 4:37 PM, Zhou, Richard <ri...@intel.com>
wrote:
> I have tested the import into hive via Kite connector as Parquet file.
> Here are some hints.
>
> 1. hive-*.jar (hive-core.jar, hive-exec.jar etc.) are needed. You
> could config the common.loader parameter in file
> server/conf/catalina.properties, or copy jars directly into folder
> server/webapps/sqoop/WEB-INF/lib/.
>
> 2. libfb303-0.9.0.jar, datanucleus-api-jdo-3.2.6.jar,
> datanucleus-core-3.2.10.jar, datanucleus-rdbms-3.2.9.jar are also needed.
>
> 3. Kite link information:
>
> sqoop:000> show link -lid 4
>
> 1 link(s) to show:
>
> link with id 4 and name kite-hive (Enabled: true, Created by root at
> 7/8/15 11:21 AM, Updated by root at 7/8/15 12:45 PM)
>
> Using Connector kite-connector with id 2
>
> Link Configuration
>
> HDFS host and port: server-654:9083
>
> 4. Job information:
>
> sqoop:000> show job -jid 6
>
> 1 job(s) to show:
>
> Job with id 6 and name mysql->kite-hive (Enabled: true, Created by root at
> 7/8/15 11:22 AM, Updated by root at 7/8/15 3:31 PM)
>
> Throttling resources
>
> Extractors:
>
> Loaders:
>
> From link: 2
>
> From database configuration
>
> Schema name:
>
> Table name: test
>
> Table SQL statement:
>
> Table column names:
>
> Partition column name: id
>
> Null value allowed for the partition column:
>
> Boundary query:
>
> Incremental read
>
> Check column:
>
> Last value:
>
> To link: 4
>
> To Kite Dataset Configuration
>
> Dataset URI: dataset:hive:tmp/aaa
>
> File format: PARQUET
>
> 5. After imported, use beeline: beeline -u
> 'jdbc:hive2://server-654:10000/tmp' -n root -p 123456
>
> 6. The data is correct
>
> 0: jdbc:hive2://server-654:10000/tmp> show tables;
>
> +-----------+--+
>
> | tab_name |
>
> +-----------+--+
>
> | aaa |
>
> +-----------+--+
>
> 1 row selected (0.182 seconds)
>
>
>
>
>
> Regards
>
> Richard
>
>
>
> *From:* Zhou, Richard [mailto:richard.zhou@intel.com]
> *Sent:* Wednesday, July 08, 2015 10:27 AM
> *To:* user@sqoop.apache.org
> *Subject:* RE: Does sqoop2 surpport import from database to hive?
>
>
>
> It is supported in 1.99.6, which has already released.
>
> You could use Kite connector to import data to hive.
>
>
>
>
>
> Regards
>
> Richard
>
>
>
> *From:* Lee S [mailto:sleefd@gmail.com <sl...@gmail.com>]
> *Sent:* Wednesday, July 08, 2015 10:18 AM
> *To:* user@sqoop.apache.org
> *Subject:* Re: Does sqoop2 surpport import from database to hive?
>
>
>
> It's fixed on version 1.99.6,when will 1.99.6 be released?
>
>
>
> On Wed, Jul 8, 2015 at 6:32 AM, Abraham Elmahrek <ab...@cloudera.com> wrote:
>
> For Sqoop2, check out this Jira:
> https://issues.apache.org/jira/browse/SQOOP-1998.
>
>
>
> If you're using the latest version of Sqoop, it's included.
>
>
>
> -Abe
>
>
>
> On Tue, Jul 7, 2015 at 3:19 AM, rudra tripathy <ru...@yahoo.com> wrote:
>
> Hi Leo,
>
>
>
> There is no meaning to import data to hive as hive understand only HDFS to
> load and one has to load data into hdfs.
>
>
>
>
>
> <font face="arial black">Thanks and Regards,</font>
>
> <font face="arial black">Rudra Prasad Tripathy</font>
>
> <strong>Sr Technical Architect</strong>
>
> <strong></strong>
>
> <strong></strong>
>
>
>
>
>
> On Tuesday, 7 July 2015 3:30 PM, Lee S <sl...@gmail.com> wrote:
>
>
>
> Hi all:
>
> Does hive jdbc can be used as a jdbc connector which can support
> importing data to hive?
>
>
>
>
>
>
>
RE: Does sqoop2 surpport import from database to hive?
Posted by "Zhou, Richard" <ri...@intel.com>.
I have tested the import into hive via Kite connector as Parquet file. Here are some hints.
1. hive-*.jar (hive-core.jar, hive-exec.jar etc.) are needed. You could config the common.loader parameter in file server/conf/catalina.properties, or copy jars directly into folder server/webapps/sqoop/WEB-INF/lib/.
2. libfb303-0.9.0.jar, datanucleus-api-jdo-3.2.6.jar, datanucleus-core-3.2.10.jar, datanucleus-rdbms-3.2.9.jar are also needed.
3. Kite link information:
sqoop:000> show link -lid 4
1 link(s) to show:
link with id 4 and name kite-hive (Enabled: true, Created by root at 7/8/15 11:21 AM, Updated by root at 7/8/15 12:45 PM)
Using Connector kite-connector with id 2
Link Configuration
HDFS host and port: server-654:9083
4. Job information:
sqoop:000> show job -jid 6
1 job(s) to show:
Job with id 6 and name mysql->kite-hive (Enabled: true, Created by root at 7/8/15 11:22 AM, Updated by root at 7/8/15 3:31 PM)
Throttling resources
Extractors:
Loaders:
From link: 2
From database configuration
Schema name:
Table name: test
Table SQL statement:
Table column names:
Partition column name: id
Null value allowed for the partition column:
Boundary query:
Incremental read
Check column:
Last value:
To link: 4
To Kite Dataset Configuration
Dataset URI: dataset:hive:tmp/aaa
File format: PARQUET
5. After imported, use beeline: beeline -u 'jdbc:hive2://server-654:10000/tmp' -n root -p 123456
6. The data is correct
0: jdbc:hive2://server-654:10000/tmp> show tables;
+-----------+--+
| tab_name |
+-----------+--+
| aaa |
+-----------+--+
1 row selected (0.182 seconds)
Regards
Richard
From: Zhou, Richard [mailto:richard.zhou@intel.com]
Sent: Wednesday, July 08, 2015 10:27 AM
To: user@sqoop.apache.org
Subject: RE: Does sqoop2 surpport import from database to hive?
It is supported in 1.99.6, which has already released.
You could use Kite connector to import data to hive.
Regards
Richard
From: Lee S [mailto:sleefd@gmail.com]
Sent: Wednesday, July 08, 2015 10:18 AM
To: user@sqoop.apache.org<ma...@sqoop.apache.org>
Subject: Re: Does sqoop2 surpport import from database to hive?
It's fixed on version 1.99.6,when will 1.99.6 be released?
On Wed, Jul 8, 2015 at 6:32 AM, Abraham Elmahrek <ab...@cloudera.com>> wrote:
For Sqoop2, check out this Jira: https://issues.apache.org/jira/browse/SQOOP-1998.
If you're using the latest version of Sqoop, it's included.
-Abe
On Tue, Jul 7, 2015 at 3:19 AM, rudra tripathy <ru...@yahoo.com>> wrote:
Hi Leo,
There is no meaning to import data to hive as hive understand only HDFS to load and one has to load data into hdfs.
<font face="arial black">Thanks and Regards,</font>
<font face="arial black">Rudra Prasad Tripathy</font>
<strong>Sr Technical Architect</strong>
<strong></strong>
<strong></strong>
On Tuesday, 7 July 2015 3:30 PM, Lee S <sl...@gmail.com>> wrote:
Hi all:
Does hive jdbc can be used as a jdbc connector which can support importing data to hive?
RE: Does sqoop2 surpport import from database to hive?
Posted by "Zhou, Richard" <ri...@intel.com>.
It is supported in 1.99.6, which has already released.
You could use Kite connector to import data to hive.
Regards
Richard
From: Lee S [mailto:sleefd@gmail.com]
Sent: Wednesday, July 08, 2015 10:18 AM
To: user@sqoop.apache.org
Subject: Re: Does sqoop2 surpport import from database to hive?
It's fixed on version 1.99.6,when will 1.99.6 be released?
On Wed, Jul 8, 2015 at 6:32 AM, Abraham Elmahrek <ab...@cloudera.com>> wrote:
For Sqoop2, check out this Jira: https://issues.apache.org/jira/browse/SQOOP-1998.
If you're using the latest version of Sqoop, it's included.
-Abe
On Tue, Jul 7, 2015 at 3:19 AM, rudra tripathy <ru...@yahoo.com>> wrote:
Hi Leo,
There is no meaning to import data to hive as hive understand only HDFS to load and one has to load data into hdfs.
<font face="arial black">Thanks and Regards,</font>
<font face="arial black">Rudra Prasad Tripathy</font>
<strong>Sr Technical Architect</strong>
<strong></strong>
<strong></strong>
On Tuesday, 7 July 2015 3:30 PM, Lee S <sl...@gmail.com>> wrote:
Hi all:
Does hive jdbc can be used as a jdbc connector which can support importing data to hive?
Re: Does sqoop2 surpport import from database to hive?
Posted by Lee S <sl...@gmail.com>.
It's fixed on version 1.99.6,when will 1.99.6 be released?
On Wed, Jul 8, 2015 at 6:32 AM, Abraham Elmahrek <ab...@cloudera.com> wrote:
> For Sqoop2, check out this Jira:
> https://issues.apache.org/jira/browse/SQOOP-1998.
>
> If you're using the latest version of Sqoop, it's included.
>
> -Abe
>
> On Tue, Jul 7, 2015 at 3:19 AM, rudra tripathy <ru...@yahoo.com> wrote:
>
>> Hi Leo,
>>
>> There is no meaning to import data to hive as hive understand only HDFS
>> to load and one has to load data into hdfs.
>>
>>
>> <font face="arial black">Thanks and Regards,</font>
>> <font face="arial black">Rudra Prasad Tripathy</font>
>> <strong>Sr Technical Architect</strong>
>> <strong></strong>
>> <strong></strong>
>>
>>
>>
>> On Tuesday, 7 July 2015 3:30 PM, Lee S <sl...@gmail.com> wrote:
>>
>>
>> Hi all:
>> Does hive jdbc can be used as a jdbc connector which can support
>> importing data to hive?
>>
>>
>>
>
Re: Does sqoop2 surpport import from database to hive?
Posted by Abraham Elmahrek <ab...@cloudera.com>.
For Sqoop2, check out this Jira:
https://issues.apache.org/jira/browse/SQOOP-1998.
If you're using the latest version of Sqoop, it's included.
-Abe
On Tue, Jul 7, 2015 at 3:19 AM, rudra tripathy <ru...@yahoo.com> wrote:
> Hi Leo,
>
> There is no meaning to import data to hive as hive understand only HDFS to
> load and one has to load data into hdfs.
>
>
> <font face="arial black">Thanks and Regards,</font>
> <font face="arial black">Rudra Prasad Tripathy</font>
> <strong>Sr Technical Architect</strong>
> <strong></strong>
> <strong></strong>
>
>
>
> On Tuesday, 7 July 2015 3:30 PM, Lee S <sl...@gmail.com> wrote:
>
>
> Hi all:
> Does hive jdbc can be used as a jdbc connector which can support
> importing data to hive?
>
>
>
Re: Does sqoop2 surpport import from database to hive?
Posted by rudra tripathy <ru...@yahoo.com>.
Hi Leo,
There is no meaning to import data to hive as hive understand only HDFS to load and one has to load data into hdfs.
<font face="arial black">Thanks and Regards,</font><font face="arial black">Rudra Prasad Tripathy</font> <strong>Sr Technical Architect</strong> <strong></strong> <strong></strong>
On Tuesday, 7 July 2015 3:30 PM, Lee S <sl...@gmail.com> wrote:
Hi all: Does hive jdbc can be used as a jdbc connector which can support importing data to hive?