You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Omkar Joshi <Om...@lntinfotech.com> on 2013/07/18 07:04:58 UTC
Export to RDBMS directly
Hi,
Currently, I'm executing the following steps(Hadoop 1.1.2, Hive 0.11 and Sqoop-1.4.3.bin__hadoop-1.0.0) :
1. Import data from MySQL to Hive using Sqoop
2. Execute a query in Hive and store its output in a Hive table
3. Export the output to MySQL using Sqoop
I was wondering if it would be possible to combine steps 2 & 3 - the output of the Hive query written directly to the MySQL database. I read about the external tables but couldn't find an example where the LOCATION clause points to something like jdbc:myql://localhost:3306/<mysql-schema>/<some-mysql-table-name>. Is it really possible?
Regards,
Omkar Joshi
________________________________
The contents of this e-mail and any attachment(s) may contain confidential or privileged information for the intended recipient(s). Unintended recipients are prohibited from taking action on the basis of information in this e-mail and using or disseminating the information, and must notify the sender and delete it from their system. L&T Infotech will not accept responsibility or liability for the accuracy or completeness of, or the presence of any virus or disabling code in this e-mail"
Re: Export to RDBMS directly
Posted by Siddharth Tiwari <si...@live.com>.
Why dont you try implementing that and contributing it to community ?
Sent from my iPhone
On Jul 17, 2013, at 10:18 PM, "Omkar Joshi" <Om...@lntinfotech.com> wrote:
> I read of the term ‘JDBC Storage Handler’ at https://issues.apache.org/jira/browse/HIVE-1555
>
> The issues seems open but I just want to confirm that it has not been implemented in the latest Hive releases.
>
> Regards,
> Omkar Joshi
>
> From: Bertrand Dechoux [mailto:dechouxb@gmail.com]
> Sent: Thursday, July 18, 2013 10:43 AM
> To: user@hive.apache.org
> Subject: Re: Export to RDBMS directly
>
> The short answer is no. You could at the moment write your own input format/output format in order to do so. I don't know all the details for hive but that's possible. However, you will likely run a DOS against your database if you are not careful. Hive could embed sqoop in order do that smartly for you but that's not the case and I doubt that is it a feature planned in the short term. You have to manage yourself the steps (for which you could use a scheduler like Oozie).
>
> Regards
> Bertrand
>
>
> On Thu, Jul 18, 2013 at 7:04 AM, Omkar Joshi <Om...@lntinfotech.com> wrote:
> Hi,
>
> Currently, I’m executing the following steps(Hadoop 1.1.2, Hive 0.11 and Sqoop-1.4.3.bin__hadoop-1.0.0) :
>
> 1. Import data from MySQL to Hive using Sqoop
>
> 2. Execute a query in Hive and store its output in a Hive table
>
> 3. Export the output to MySQL using Sqoop
>
>
> I was wondering if it would be possible to combine steps 2 & 3 – the output of the Hive query written directly to the MySQL database. I read about the external tables but couldn’t find an example where the LOCATION clause points to something like jdbc:myql://localhost:3306/<mysql-schema>/<some-mysql-table-name>. Is it really possible?
>
> Regards,
> Omkar Joshi
>
>
> The contents of this e-mail and any attachment(s) may contain confidential or privileged information for the intended recipient(s). Unintended recipients are prohibited from taking action on the basis of information in this e-mail and using or disseminating the information, and must notify the sender and delete it from their system. L&T Infotech will not accept responsibility or liability for the accuracy or completeness of, or the presence of any virus or disabling code in this e-mail"
>
>
>
> --
> Bertrand Dechoux
RE: Export to RDBMS directly
Posted by Omkar Joshi <Om...@lntinfotech.com>.
I read of the term 'JDBC Storage Handler' at https://issues.apache.org/jira/browse/HIVE-1555
The issues seems open but I just want to confirm that it has not been implemented in the latest Hive releases.
Regards,
Omkar Joshi
From: Bertrand Dechoux [mailto:dechouxb@gmail.com]
Sent: Thursday, July 18, 2013 10:43 AM
To: user@hive.apache.org
Subject: Re: Export to RDBMS directly
The short answer is no. You could at the moment write your own input format/output format in order to do so. I don't know all the details for hive but that's possible. However, you will likely run a DOS against your database if you are not careful. Hive could embed sqoop in order do that smartly for you but that's not the case and I doubt that is it a feature planned in the short term. You have to manage yourself the steps (for which you could use a scheduler like Oozie).
Regards
Bertrand
On Thu, Jul 18, 2013 at 7:04 AM, Omkar Joshi <Om...@lntinfotech.com>> wrote:
Hi,
Currently, I'm executing the following steps(Hadoop 1.1.2, Hive 0.11 and Sqoop-1.4.3.bin__hadoop-1.0.0) :
1. Import data from MySQL to Hive using Sqoop
2. Execute a query in Hive and store its output in a Hive table
3. Export the output to MySQL using Sqoop
I was wondering if it would be possible to combine steps 2 & 3 - the output of the Hive query written directly to the MySQL database. I read about the external tables but couldn't find an example where the LOCATION clause points to something like jdbc:myql://localhost:3306/<mysql-schema>/<some-mysql-table-name>. Is it really possible?
Regards,
Omkar Joshi
________________________________
The contents of this e-mail and any attachment(s) may contain confidential or privileged information for the intended recipient(s). Unintended recipients are prohibited from taking action on the basis of information in this e-mail and using or disseminating the information, and must notify the sender and delete it from their system. L&T Infotech will not accept responsibility or liability for the accuracy or completeness of, or the presence of any virus or disabling code in this e-mail"
--
Bertrand Dechoux
Re: Export to RDBMS directly
Posted by Bertrand Dechoux <de...@gmail.com>.
The short answer is no. You could at the moment write your own input
format/output format in order to do so. I don't know all the details for
hive but that's possible. However, you will likely run a DOS against your
database if you are not careful. Hive could embed sqoop in order do that
smartly for you but that's not the case and I doubt that is it a feature
planned in the short term. You have to manage yourself the steps (for which
you could use a scheduler like Oozie).
Regards
Bertrand
On Thu, Jul 18, 2013 at 7:04 AM, Omkar Joshi <Om...@lntinfotech.com>wrote:
> Hi,****
>
> ** **
>
> Currently, I’m executing the following steps(Hadoop 1.1.2, Hive 0.11 and
> Sqoop-1.4.3.bin__hadoop-1.0.0) :****
>
> ** **
>
> **1. **Import data from MySQL to Hive using Sqoop****
>
> **2. **Execute a query in Hive and store its output in a Hive table****
>
> **3. **Export the output to MySQL using Sqoop****
>
> ** **
>
> I was wondering if it would be possible to combine steps 2 & 3 – the
> output of the Hive query written directly to the MySQL database. I read
> about the external tables but couldn’t find an example where the LOCATION
> clause points to something like
> jdbc:myql://localhost:3306/<mysql-schema>/<some-mysql-table-name>. Is it
> really possible?****
>
> ** **
>
> Regards,****
>
> Omkar Joshi****
>
> ** **
>
> ------------------------------
> The contents of this e-mail and any attachment(s) may contain confidential
> or privileged information for the intended recipient(s). Unintended
> recipients are prohibited from taking action on the basis of information in
> this e-mail and using or disseminating the information, and must notify the
> sender and delete it from their system. L&T Infotech will not accept
> responsibility or liability for the accuracy or completeness of, or the
> presence of any virus or disabling code in this e-mail"
>
--
Bertrand Dechoux