You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Dmitry Tolpeko <dm...@gmail.com> on 2015/02/27 21:36:07 UTC
PL/HQL - Procedural SQL-on-Hadoop
Let me introduce PL/HQL, an open source tool that implements procedural SQL
on Hadoop. It is going to support all major procedural syntaxes. The tool
can be used with any SQL-on-Hadoop solution.
Motivation:
- Writing the driver code using well-known procedural SQL (not bash)
that enables Hadoop to even more wider audience
- Allowing dynamic SQL, iterations, flow-of-control and SQL exception
handling
- Facilitating migration of RDBMS workload to Hadoop
Plans (besides extending syntax):
- Supporting CREATE PROCEDURE/FUNCTION/PACKAGE to reuse code
- Allowing connections to multiple databases (i.e. lookup tables in
relational databases)
- On-the-fly SQL conversion (SELECT i.e.), compatibility layers
More details can be found at http://www.plhql.org/
It is just the first release PL/HQL 0.01 to show that such project exists
and get any initial feedback.
Thank you,
Dmitry Tolpeko
Re: PL/HQL - Procedural SQL-on-Hadoop
Posted by Dmitry Tolpeko <dm...@gmail.com>.
>Are there any github link for source code?
Will be added soon
>Any there any documents for writing the sproc?
Please see the reference at plhql.org. Briefly, just put your code without
enclosing CREATE PROC/FUNC statements (they will be supported soon)
>What's performance than regular HQL statements?
It is comparable as the tool holds Hive JDBC connection and just submits
queries (but faster comparing with lauching hive from bash for each query)
>Is there a plan to support spark SQL?
Now you can use any SQL-on-Hadoop configuring a proper JDBC connection.
Dmitry
On Sat, Feb 28, 2015 at 3:45 AM, DU DU <wi...@gmail.com> wrote:
> Are there any github link for source code?
> Any there any documents for writing the sproc?
> What's performance than regular HQL statements?
> Is there a plan to support spark SQL?
> will
>
> On Fri, Feb 27, 2015 at 3:36 PM, Dmitry Tolpeko <dm...@gmail.com>
> wrote:
>
>> Let me introduce PL/HQL, an open source tool that implements procedural
>> SQL on Hadoop. It is going to support all major procedural syntaxes. The
>> tool can be used with any SQL-on-Hadoop solution.
>>
>> Motivation:
>>
>> - Writing the driver code using well-known procedural SQL (not bash)
>> that enables Hadoop to even more wider audience
>> - Allowing dynamic SQL, iterations, flow-of-control and SQL exception
>> handling
>> - Facilitating migration of RDBMS workload to Hadoop
>>
>> Plans (besides extending syntax):
>>
>>
>> - Supporting CREATE PROCEDURE/FUNCTION/PACKAGE to reuse code
>> - Allowing connections to multiple databases (i.e. lookup tables in
>> relational databases)
>> - On-the-fly SQL conversion (SELECT i.e.), compatibility layers
>>
>> More details can be found at http://www.plhql.org/
>>
>> It is just the first release PL/HQL 0.01 to show that such project exists
>> and get any initial feedback.
>>
>> Thank you,
>>
>> Dmitry Tolpeko
>>
>
>
>
> --
> Thanks,
> Will
>
Re: PL/HQL - Procedural SQL-on-Hadoop
Posted by DU DU <wi...@gmail.com>.
Are there any github link for source code?
Any there any documents for writing the sproc?
What's performance than regular HQL statements?
Is there a plan to support spark SQL?
will
On Fri, Feb 27, 2015 at 3:36 PM, Dmitry Tolpeko <dm...@gmail.com> wrote:
> Let me introduce PL/HQL, an open source tool that implements procedural
> SQL on Hadoop. It is going to support all major procedural syntaxes. The
> tool can be used with any SQL-on-Hadoop solution.
>
> Motivation:
>
> - Writing the driver code using well-known procedural SQL (not bash)
> that enables Hadoop to even more wider audience
> - Allowing dynamic SQL, iterations, flow-of-control and SQL exception
> handling
> - Facilitating migration of RDBMS workload to Hadoop
>
> Plans (besides extending syntax):
>
>
> - Supporting CREATE PROCEDURE/FUNCTION/PACKAGE to reuse code
> - Allowing connections to multiple databases (i.e. lookup tables in
> relational databases)
> - On-the-fly SQL conversion (SELECT i.e.), compatibility layers
>
> More details can be found at http://www.plhql.org/
>
> It is just the first release PL/HQL 0.01 to show that such project exists
> and get any initial feedback.
>
> Thank you,
>
> Dmitry Tolpeko
>
--
Thanks,
Will
Re: PL/HQL - Procedural SQL-on-Hadoop
Posted by Dmitry Tolpeko <dm...@gmail.com>.
Venkat,
The goal of this project is to execute existing PL/SQL in Hive as much as
possible, not to migrate. In case when some design restrictions are faced
the code has to be redesigned, but hopefully most of the remaining code
remained untouched, no need to convert everything to bash/Python etc.
Dmitry
On Tue, Mar 3, 2015 at 4:39 AM, Venkat, Ankam <An...@centurylink.com>
wrote:
> Is there a simple way to migrate from PL/SQL to PL/HQL?
>
>
>
> Regards,
>
> Venkat
>
>
>
> *From:* Dmitry Tolpeko [mailto:dmtolpeko@gmail.com]
> *Sent:* Friday, February 27, 2015 1:36 PM
> *To:* user@hive.apache.org
> *Subject:* PL/HQL - Procedural SQL-on-Hadoop
>
>
>
> Let me introduce PL/HQL, an open source tool that implements procedural
> SQL on Hadoop. It is going to support all major procedural syntaxes. The
> tool can be used with any SQL-on-Hadoop solution.
>
>
>
> Motivation:
>
> - Writing the driver code using well-known procedural SQL (not bash)
> that enables Hadoop to even more wider audience
> - Allowing dynamic SQL, iterations, flow-of-control and SQL exception
> handling
> - Facilitating migration of RDBMS workload to Hadoop
>
> Plans (besides extending syntax):
>
>
>
> - Supporting CREATE PROCEDURE/FUNCTION/PACKAGE to reuse code
> - Allowing connections to multiple databases (i.e. lookup tables in
> relational databases)
> - On-the-fly SQL conversion (SELECT i.e.), compatibility layers
>
> More details can be found at http://www.plhql.org/
>
>
>
> It is just the first release PL/HQL 0.01 to show that such project exists
> and get any initial feedback.
>
>
>
> Thank you,
>
>
>
> Dmitry Tolpeko
> This communication is the property of CenturyLink and may contain
> confidential or privileged information. Unauthorized use of this
> communication is strictly prohibited and may be unlawful. If you have
> received this communication in error, please immediately notify the sender
> by reply e-mail and destroy all copies of the communication and any
> attachments.
>
RE: PL/HQL - Procedural SQL-on-Hadoop
Posted by "Venkat, Ankam" <An...@centurylink.com>.
Is there a simple way to migrate from PL/SQL to PL/HQL?
Regards,
Venkat
From: Dmitry Tolpeko [mailto:dmtolpeko@gmail.com]
Sent: Friday, February 27, 2015 1:36 PM
To: user@hive.apache.org
Subject: PL/HQL - Procedural SQL-on-Hadoop
Let me introduce PL/HQL, an open source tool that implements procedural SQL on Hadoop. It is going to support all major procedural syntaxes. The tool can be used with any SQL-on-Hadoop solution.
Motivation:
* Writing the driver code using well-known procedural SQL (not bash) that enables Hadoop to even more wider audience
* Allowing dynamic SQL, iterations, flow-of-control and SQL exception handling
* Facilitating migration of RDBMS workload to Hadoop
Plans (besides extending syntax):
* Supporting CREATE PROCEDURE/FUNCTION/PACKAGE to reuse code
* Allowing connections to multiple databases (i.e. lookup tables in relational databases)
* On-the-fly SQL conversion (SELECT i.e.), compatibility layers
More details can be found at http://www.plhql.org/
It is just the first release PL/HQL 0.01 to show that such project exists and get any initial feedback.
Thank you,
Dmitry Tolpeko
This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.
Re: Re: PL/HQL - Procedural SQL-on-Hadoop
Posted by "r7raul1984@163.com" <r7...@163.com>.
You can refer to the cursor columns using cur_name.col_name syntax
Great job!
r7raul1984@163.com
From: Dmitry Tolpeko
Date: 2015-05-01 03:43
To: user
Subject: Re: PL/HQL - Procedural SQL-on-Hadoop
Hi everyone,
PL/HQL 0.3.1 released today. There are some features I hope you find interesting:
1) Stored procedures and SQL user-defined functions
CREATE PROCEDURE and CREATE FUNCTION statements are supported so you can define procedures and functions in your scripts, and when added to .plhqlrc they can look as real stored objects.
2) Calling PL/HQL functions from Hive queries
You can use PL/HQL functions in Hive queries. PL/HQL automatically puts function definitions to distributed cache, registers Hive UDF and executes functions. You can also call PL/HQL functions from Hive CLI if you manually register UDF and execute ADD FILE for PL/HQL scripts. See, http://plhql.org/udf
3) Multiple connections
You can work with multiple databases from a single PL/HQL script. For example, run queries in Hive, but lookup and log to RDBMS.
MAP OBJECT statement allows you to assign a table to a connection (again you can do that in plhqlrc). Then PL/HQL will automatically use appropriate connection.
In the next releases, procedural features will be extended to cover more existing dialects, on-the-fly SQL conversion will be improved. Also federation features will be added - like saving query results to RDBMS (actually any JDBC source), use RDBMS tables as map-join tables in Hive queries and so on.
Early adopters are very welcome. Please contact me at dmtolpeko@gmail.com
Thanks,
Dmitry Tolpeko
PL/HQL - Procedural SQL-on-Hadoop, www.plhql.org
On Fri, Mar 6, 2015 at 6:23 PM, Ronak Bhatt <ro...@gmail.com> wrote:
plhql is a great initiative and will certainly help with more adoption of the eco-system...Good job Dimtry
-r
-r
On Fri, Mar 6, 2015 at 1:07 AM, Dmitry Tolpeko <dm...@gmail.com> wrote:
run dos2unix for plhql file. Sorry for this issue, it will be fixed.
Dmitry
On Fri, Mar 6, 2015 at 11:27 AM, r7raul1984@163.com <r7...@163.com> wrote:
I type plhql --version .... but meet error bash: /usr/plhql/plhql: /bin/bash^M: bad interpreter: No such file or directory
r7raul1984@163.com
From: Dmitry Tolpeko
Date: 2015-02-28 04:36
To: user
Subject: PL/HQL - Procedural SQL-on-Hadoop
Let me introduce PL/HQL, an open source tool that implements procedural SQL on Hadoop. It is going to support all major procedural syntaxes. The tool can be used with any SQL-on-Hadoop solution.
Motivation:
Writing the driver code using well-known procedural SQL (not bash) that enables Hadoop to even more wider audience
Allowing dynamic SQL, iterations, flow-of-control and SQL exception handling
Facilitating migration of RDBMS workload to Hadoop
Plans (besides extending syntax):
Supporting CREATE PROCEDURE/FUNCTION/PACKAGE to reuse code
Allowing connections to multiple databases (i.e. lookup tables in relational databases)
On-the-fly SQL conversion (SELECT i.e.), compatibility layers
More details can be found at http://www.plhql.org/
It is just the first release PL/HQL 0.01 to show that such project exists and get any initial feedback.
Thank you,
Dmitry Tolpeko
Re: PL/HQL - Procedural SQL-on-Hadoop
Posted by Dmitry Tolpeko <dm...@gmail.com>.
Hi everyone,
PL/HQL 0.3.1 released today. There are some features I hope you find
interesting:
1) Stored procedures and SQL user-defined functions
CREATE PROCEDURE and CREATE FUNCTION statements are supported so you can
define procedures and functions in your scripts, and when added to .plhqlrc
they can look as real stored objects.
2) Calling PL/HQL functions from Hive queries
You can use PL/HQL functions in Hive queries. PL/HQL automatically puts
function definitions to distributed cache, registers Hive UDF and executes
functions. You can also call PL/HQL functions from Hive CLI if you manually
register UDF and execute ADD FILE for PL/HQL scripts. See,
http://plhql.org/udf
3) Multiple connections
You can work with multiple databases from a single PL/HQL script. For
example, run queries in Hive, but lookup and log to RDBMS.
MAP OBJECT statement allows you to assign a table to a connection (again
you can do that in plhqlrc). Then PL/HQL will automatically use appropriate
connection.
In the next releases, procedural features will be extended to cover more
existing dialects, on-the-fly SQL conversion will be improved. Also
federation features will be added - like saving query results to RDBMS
(actually any JDBC source), use RDBMS tables as map-join tables in Hive
queries and so on.
Early adopters are very welcome. Please contact me at dmtolpeko@gmail.com
Thanks,
Dmitry Tolpeko
PL/HQL - Procedural SQL-on-Hadoop, www.plhql.org
On Fri, Mar 6, 2015 at 6:23 PM, Ronak Bhatt <ro...@gmail.com> wrote:
> plhql is a great initiative and will certainly help with more adoption of
> the eco-system...Good job Dimtry
>
> -r
>
> *-r*
>
>
>
>
> On Fri, Mar 6, 2015 at 1:07 AM, Dmitry Tolpeko <dm...@gmail.com>
> wrote:
>
>> run dos2unix for plhql file. Sorry for this issue, it will be fixed.
>>
>> Dmitry
>>
>> On Fri, Mar 6, 2015 at 11:27 AM, r7raul1984@163.com <r7...@163.com>
>> wrote:
>>
>>> I type plhql --version .... but meet error bash: /usr/plhql/plhql:
>>> /bin/bash^M: bad interpreter: No such file or directory
>>>
>>> ------------------------------
>>> r7raul1984@163.com
>>>
>>>
>>> *From:* Dmitry Tolpeko <dm...@gmail.com>
>>> *Date:* 2015-02-28 04:36
>>> *To:* user <us...@hive.apache.org>
>>> *Subject:* PL/HQL - Procedural SQL-on-Hadoop
>>> Let me introduce PL/HQL, an open source tool that implements procedural
>>> SQL on Hadoop. It is going to support all major procedural syntaxes. The
>>> tool can be used with any SQL-on-Hadoop solution.
>>>
>>> Motivation:
>>>
>>> - Writing the driver code using well-known procedural SQL (not bash)
>>> that enables Hadoop to even more wider audience
>>> - Allowing dynamic SQL, iterations, flow-of-control and SQL
>>> exception handling
>>> - Facilitating migration of RDBMS workload to Hadoop
>>>
>>> Plans (besides extending syntax):
>>>
>>>
>>> - Supporting CREATE PROCEDURE/FUNCTION/PACKAGE to reuse code
>>> - Allowing connections to multiple databases (i.e. lookup tables in
>>> relational databases)
>>> - On-the-fly SQL conversion (SELECT i.e.), compatibility layers
>>>
>>> More details can be found at http://www.plhql.org/
>>>
>>> It is just the first release PL/HQL 0.01 to show that such project
>>> exists and get any initial feedback.
>>>
>>> Thank you,
>>>
>>> Dmitry Tolpeko
>>>
>>>
>>
>
Re: PL/HQL - Procedural SQL-on-Hadoop
Posted by Ronak Bhatt <ro...@gmail.com>.
plhql is a great initiative and will certainly help with more adoption of
the eco-system...Good job Dimtry
-r
*-r*
On Fri, Mar 6, 2015 at 1:07 AM, Dmitry Tolpeko <dm...@gmail.com> wrote:
> run dos2unix for plhql file. Sorry for this issue, it will be fixed.
>
> Dmitry
>
> On Fri, Mar 6, 2015 at 11:27 AM, r7raul1984@163.com <r7...@163.com>
> wrote:
>
>> I type plhql --version .... but meet error bash: /usr/plhql/plhql:
>> /bin/bash^M: bad interpreter: No such file or directory
>>
>> ------------------------------
>> r7raul1984@163.com
>>
>>
>> *From:* Dmitry Tolpeko <dm...@gmail.com>
>> *Date:* 2015-02-28 04:36
>> *To:* user <us...@hive.apache.org>
>> *Subject:* PL/HQL - Procedural SQL-on-Hadoop
>> Let me introduce PL/HQL, an open source tool that implements procedural
>> SQL on Hadoop. It is going to support all major procedural syntaxes. The
>> tool can be used with any SQL-on-Hadoop solution.
>>
>> Motivation:
>>
>> - Writing the driver code using well-known procedural SQL (not bash)
>> that enables Hadoop to even more wider audience
>> - Allowing dynamic SQL, iterations, flow-of-control and SQL exception
>> handling
>> - Facilitating migration of RDBMS workload to Hadoop
>>
>> Plans (besides extending syntax):
>>
>>
>> - Supporting CREATE PROCEDURE/FUNCTION/PACKAGE to reuse code
>> - Allowing connections to multiple databases (i.e. lookup tables in
>> relational databases)
>> - On-the-fly SQL conversion (SELECT i.e.), compatibility layers
>>
>> More details can be found at http://www.plhql.org/
>>
>> It is just the first release PL/HQL 0.01 to show that such project exists
>> and get any initial feedback.
>>
>> Thank you,
>>
>> Dmitry Tolpeko
>>
>>
>
Re: PL/HQL - Procedural SQL-on-Hadoop
Posted by Dmitry Tolpeko <dm...@gmail.com>.
run dos2unix for plhql file. Sorry for this issue, it will be fixed.
Dmitry
On Fri, Mar 6, 2015 at 11:27 AM, r7raul1984@163.com <r7...@163.com>
wrote:
> I type plhql --version .... but meet error bash: /usr/plhql/plhql:
> /bin/bash^M: bad interpreter: No such file or directory
>
> ------------------------------
> r7raul1984@163.com
>
>
> *From:* Dmitry Tolpeko <dm...@gmail.com>
> *Date:* 2015-02-28 04:36
> *To:* user <us...@hive.apache.org>
> *Subject:* PL/HQL - Procedural SQL-on-Hadoop
> Let me introduce PL/HQL, an open source tool that implements procedural
> SQL on Hadoop. It is going to support all major procedural syntaxes. The
> tool can be used with any SQL-on-Hadoop solution.
>
> Motivation:
>
> - Writing the driver code using well-known procedural SQL (not bash)
> that enables Hadoop to even more wider audience
> - Allowing dynamic SQL, iterations, flow-of-control and SQL exception
> handling
> - Facilitating migration of RDBMS workload to Hadoop
>
> Plans (besides extending syntax):
>
>
> - Supporting CREATE PROCEDURE/FUNCTION/PACKAGE to reuse code
> - Allowing connections to multiple databases (i.e. lookup tables in
> relational databases)
> - On-the-fly SQL conversion (SELECT i.e.), compatibility layers
>
> More details can be found at http://www.plhql.org/
>
> It is just the first release PL/HQL 0.01 to show that such project exists
> and get any initial feedback.
>
> Thank you,
>
> Dmitry Tolpeko
>
>
Re: PL/HQL - Procedural SQL-on-Hadoop
Posted by "r7raul1984@163.com" <r7...@163.com>.
I type plhql --version .... but meet error bash: /usr/plhql/plhql: /bin/bash^M: bad interpreter: No such file or directory
r7raul1984@163.com
From: Dmitry Tolpeko
Date: 2015-02-28 04:36
To: user
Subject: PL/HQL - Procedural SQL-on-Hadoop
Let me introduce PL/HQL, an open source tool that implements procedural SQL on Hadoop. It is going to support all major procedural syntaxes. The tool can be used with any SQL-on-Hadoop solution.
Motivation:
Writing the driver code using well-known procedural SQL (not bash) that enables Hadoop to even more wider audience
Allowing dynamic SQL, iterations, flow-of-control and SQL exception handling
Facilitating migration of RDBMS workload to Hadoop
Plans (besides extending syntax):
Supporting CREATE PROCEDURE/FUNCTION/PACKAGE to reuse code
Allowing connections to multiple databases (i.e. lookup tables in relational databases)
On-the-fly SQL conversion (SELECT i.e.), compatibility layers
More details can be found at http://www.plhql.org/
It is just the first release PL/HQL 0.01 to show that such project exists and get any initial feedback.
Thank you,
Dmitry Tolpeko