You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Ning Zhang <nz...@fb.com> on 2011/02/08 20:38:47 UTC
Review Request: HIVE-818. Create a Hive CLI that connects to hive
ThriftServer
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/407/
-----------------------------------------------------------
Review request for hive.
Summary
-------
Copied from JIRA HIVE-818:
This patch does the following:
add 2 options (-h, -p) in CLI to specify the hostname and port of Hive server.
change the HiveServer to output non-Hive commands (non Driver) to a temp file and change the fetchOne/fetchN/fetchAll functions to get results from the temp file.
change the fetchOne function to throw a HiveServerException (error code 0) when reaching the end of result rather than sending an empty string.
Caveats:
session.err from the HiveServer is still not sending back to client. So the progress of a Hadoop job is not shown in the client side in remote mode (I think there is a JIRA opened already. If not I wil file a follow-up JIRA for this).
now end-to-end unit test for remote mode. I manually tested HiveServer and CLI in remote mode (set/dfs/SQL commands) and in combination of -e/-f options. I will file a follow-up JIRA for creating a unit test suite for remote mode CLI.
Diffs
-----
trunk/build.xml 1068272
trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 1068272
trunk/cli/src/java/org/apache/hadoop/hive/cli/CliSessionState.java 1068272
trunk/cli/src/java/org/apache/hadoop/hive/cli/OptionsProcessor.java 1068272
trunk/ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java 1068272
trunk/service/if/hive_service.thrift 1068272
trunk/service/src/gen/thrift/gen-cpp/ThriftHive.h 1068272
trunk/service/src/gen/thrift/gen-cpp/ThriftHive.cpp 1068272
trunk/service/src/gen/thrift/gen-cpp/ThriftHive_server.skeleton.cpp 1068272
trunk/service/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/service/ThriftHive.java 1068272
trunk/service/src/gen/thrift/gen-php/hive_service/ThriftHive.php 1068272
trunk/service/src/gen/thrift/gen-py/hive_service/ThriftHive-remote 1068272
trunk/service/src/gen/thrift/gen-py/hive_service/ThriftHive.py 1068272
trunk/service/src/gen/thrift/gen-rb/thrift_hive.rb 1068272
trunk/service/src/java/org/apache/hadoop/hive/service/HiveServer.java 1068272
Diff: https://reviews.apache.org/r/407/diff
Testing
-------
Passed all unit tests.
Also manually tested HiveServer and CLI remote mode by:
1) $ hive --service hiveserver
2) in another terminal: hive -h localhost
3) tested the following command:
- set; -- get all parameters
- set hive.stats.autogather; -- check default parameter value
- set hive.stats.autogather=false; -- change parameter value
- set hive.stats.autogather; -- check parameter value got changed
- select * from src; -- Hive query but no Hadoop job
- select count(*) from src; -- Hive query and Hadoop job is created
- select k from src; -- negative test case where SemanticAnalyzer throw an exception
- show partitions srcpart; -- Hive Query but no hadoop job
- explain select count(*) from srcpart where ds is not null; -- explain query
Thanks,
Ning
Re: Review Request: HIVE-818. Create a Hive CLI that connects to hive
ThriftServer
Posted by Ning Zhang <nz...@fb.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/407/
-----------------------------------------------------------
(Updated 2011-02-09 15:11:19.773829)
Review request for hive.
Changes
-------
Updated the diff according to the new patch.
Summary
-------
Copied from JIRA HIVE-818:
This patch does the following:
add 2 options (-h, -p) in CLI to specify the hostname and port of Hive server.
change the HiveServer to output non-Hive commands (non Driver) to a temp file and change the fetchOne/fetchN/fetchAll functions to get results from the temp file.
change the fetchOne function to throw a HiveServerException (error code 0) when reaching the end of result rather than sending an empty string.
Caveats:
session.err from the HiveServer is still not sending back to client. So the progress of a Hadoop job is not shown in the client side in remote mode (I think there is a JIRA opened already. If not I wil file a follow-up JIRA for this).
now end-to-end unit test for remote mode. I manually tested HiveServer and CLI in remote mode (set/dfs/SQL commands) and in combination of -e/-f options. I will file a follow-up JIRA for creating a unit test suite for remote mode CLI.
Diffs (updated)
-----
trunk/build.xml 1069164
trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java 1069164
trunk/cli/src/java/org/apache/hadoop/hive/cli/CliSessionState.java 1069164
trunk/cli/src/java/org/apache/hadoop/hive/cli/OptionsProcessor.java 1069164
trunk/ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java 1069164
trunk/service/if/hive_service.thrift 1069164
trunk/service/src/gen/thrift/gen-cpp/ThriftHive.h 1069164
trunk/service/src/gen/thrift/gen-cpp/ThriftHive.cpp 1069164
trunk/service/src/gen/thrift/gen-cpp/ThriftHive_server.skeleton.cpp 1069164
trunk/service/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/service/ThriftHive.java 1069164
trunk/service/src/gen/thrift/gen-php/hive_service/ThriftHive.php 1069164
trunk/service/src/gen/thrift/gen-py/hive_service/ThriftHive-remote 1069164
trunk/service/src/gen/thrift/gen-py/hive_service/ThriftHive.py 1069164
trunk/service/src/gen/thrift/gen-rb/thrift_hive.rb 1069164
trunk/service/src/java/org/apache/hadoop/hive/service/HiveServer.java 1069164
Diff: https://reviews.apache.org/r/407/diff
Testing
-------
Passed all unit tests.
Also manually tested HiveServer and CLI remote mode by:
1) $ hive --service hiveserver
2) in another terminal: hive -h localhost
3) tested the following command:
- set; -- get all parameters
- set hive.stats.autogather; -- check default parameter value
- set hive.stats.autogather=false; -- change parameter value
- set hive.stats.autogather; -- check parameter value got changed
- select * from src; -- Hive query but no Hadoop job
- select count(*) from src; -- Hive query and Hadoop job is created
- select k from src; -- negative test case where SemanticAnalyzer throw an exception
- show partitions srcpart; -- Hive Query but no hadoop job
- explain select count(*) from srcpart where ds is not null; -- explain query
Thanks,
Ning