You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tajo.apache.org by "JaeHwa Jung (JIRA)" <ji...@apache.org> on 2013/09/08 20:03:51 UTC
[jira] [Updated] (TAJO-101) HiveQL converter
[ https://issues.apache.org/jira/browse/TAJO-101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
JaeHwa Jung updated TAJO-101:
-----------------------------
Attachment: TAJO-101.patch
HiveConverter converts Hive query ANSI SQL for Tajo.
If user write Hive query, HiveConverter parse for the first time.
Then HiveConverter makes algebra datas to execute it on Tajo.
If you want to use HiveConverter, You have to update tajo-site.xml as follows:
{code:xml}
<property>
<name>tajo.hive.query.mode</name>
<value>true</value>
</property>
{code}
But in this case, you must restart your Tajo master. I think that it is inconvenient
.
So I updated GlobalEngine. If you write on Tajo cli as follows:
{code:xml}
set tajo.hive.query.mode=true;
{code}
You can write Hive Query. And If you write ANSI SQL on Tajo, then you have to write as follows:
{code:xml}
set tajo.hive.query.mode=false;
{code}
For reference, this is my test cases.
{code:xml}
tajo> select * from table1;
2013-09-09 02:46:05,533 INFO client.TajoClient (TajoClient.java:connectionToQueryMaster(190)) - Connected to Query Master (qid=q_1378662327647_0001, addr=192.168.1.103:20200)
Progress: 0%, response time: 0.417 sec
Progress: 0%, response time: 1.42 sec
Progress: 0%, response time: 2.422 sec
Progress: 0%, response time: 3.424 sec
Progress: 0%, response time: 4.426 sec
Progress: 0%, response time: 5.428 sec
Progress: 100%, response time: 6.44 sec
Progress: 100%, response time: 6.455 sec
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/blrunner/tajo-0.2.0-SNAPSHOT/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/blrunner/hadoop-2.0.3-alpha/share/hadoop/common/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
2013-09-09 02:46:12.730 java[19355:1203] Unable to load realm info from SCDynamicStore
2013-09-09 02:46:12,922 WARN util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2013-09-09 02:46:12.925 java[19355:1203] Unable to load realm info from SCDynamicStore
final state: QUERY_SUCCEEDED, init time: 1.395 sec, execution time: 0.0 sec, total response time: 6.455 sec
result: hdfs://localhost:9010/user/blrunner/tajo/q_1378662327647_0001
id, name, score, type
-------------------------------
1, abc, 1.1, a
2, def, 2.3, b
3, ghi, 3.4, c
4, jkl, 4.5, d
5, mno, 5.6, e
2013-09-09 02:46:13,469 INFO rpc.NettyClientBase (NettyClientBase.java:close(87)) - Proxy is disconnected from 192.168.1.103:20200
2013-09-09 02:46:13,469 INFO client.TajoClient (TajoClient.java:closeQuery(113)) - Closed a QueryMaster connection (qid=q_1378662327647_0001, addr=blrunnerui-MacBook-Pro.local/192.168.1.103:20200)
tajo> from table1 select *;
ERROR: syntax error at or near 'from'
LINE 1:0 from table1 select *
^^^^
tajo> set tajo.hive.query.mode=true;
OK
tajo> from table1 select *;
2013-09-09 02:46:38,510 INFO client.TajoClient (TajoClient.java:connectionToQueryMaster(190)) - Connected to Query Master (qid=q_1378662327647_0002, addr=192.168.1.103:10186)
Progress: 0%, response time: 0.873 sec
Progress: 0%, response time: 1.876 sec
Progress: 0%, response time: 2.877 sec
Progress: 0%, response time: 3.88 sec
Progress: 0%, response time: 4.882 sec
Progress: 0%, response time: 5.884 sec
Progress: 100%, response time: 6.568 sec
final state: QUERY_SUCCEEDED, init time: 1.242 sec, execution time: 0.0 sec, total response time: 6.568 sec
result: hdfs://localhost:9010/user/blrunner/tajo/q_1378662327647_0002
id, name, score, type
-------------------------------
1, abc, 1.1, a
2, def, 2.3, b
3, ghi, 3.4, c
4, jkl, 4.5, d
5, mno, 5.6, e
2013-09-09 02:46:44,590 INFO rpc.NettyClientBase (NettyClientBase.java:close(87)) - Proxy is disconnected from 192.168.1.103:10186
2013-09-09 02:46:44,590 INFO client.TajoClient (TajoClient.java:closeQuery(113)) - Closed a QueryMaster connection (qid=q_1378662327647_0002, addr=blrunnerui-MacBook-Pro.local/192.168.1.103:10186)
tajo> set tajo.hive.query.mode=false;
OK
tajo> from table1 select *;
ERROR: syntax error at or near 'from'
LINE 1:0 from table1 select *
{code}
> HiveQL converter
> ----------------
>
> Key: TAJO-101
> URL: https://issues.apache.org/jira/browse/TAJO-101
> Project: Tajo
> Issue Type: New Feature
> Reporter: JaeHwa Jung
> Assignee: JaeHwa Jung
> Attachments: TAJO-101.patch
>
>
> I hit a new idea when I see tajo-algebra issue. Many users use Hive to analysis big data. If Hive user can use HiveQL, they can easily use Tajo.
> For this work, I'll borrow HQL grammar from Hive and change to support ANTLR4 specification, and we should develop a HiveQL parser to use tajo-algebra.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira