You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Raghotham Murthy (JIRA)" <ji...@apache.org> on 2008/11/19 20:14:44 UTC
[jira] Created: (HIVE-73) Thrift Server and Client for Hive
Thrift Server and Client for Hive
---------------------------------
Key: HIVE-73
URL: https://issues.apache.org/jira/browse/HIVE-73
Project: Hadoop Hive
Issue Type: New Feature
Affects Versions: 0.19.0
Reporter: Raghotham Murthy
Fix For: 0.20.0
Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-73) Thrift Server and Client for Hive
Posted by "Raghotham Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghotham Murthy updated HIVE-73:
---------------------------------
Attachment: hive-73.7.patch
- now using only tableDesc from FetchTask instead of passing around colInfos
- removed buffering in HiveServer. Changed Driver/FetchTask to return the required number of rows.
- added javadocs
- added getStatus and getVersion to thrift interface
- can now run test on stand alone server by setting test.service.standalone.server in data/conf/hive-site.xml. Need to start the server by running sh service/scripts/start_hive_thrift_server.sh from the top level directory
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.2.patch, hive-73.3.txt, hive-73.4.txt, hive-73.5.txt, hive-73.6.patch, hive-73.7.patch
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-73) Thrift Server and Client for Hive
Posted by "Raghotham Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghotham Murthy updated HIVE-73:
---------------------------------
Attachment: hive-73.2.patch
attached new patch file
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.2.patch
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-73) Thrift Server and Client for Hive
Posted by "Michi Mutsuzaki (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12650432#action_12650432 ]
Michi Mutsuzaki commented on HIVE-73:
-------------------------------------
By the way, how often do you guys commit to svn? It's kind of painful and error-prone to keep passing around patches for a long time.
--Michi
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.2.patch, hive-73.3.txt, hive-73.4.txt, hive-73.5.txt
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-73) Thrift Server and Client for Hive
Posted by "Raghotham Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghotham Murthy updated HIVE-73:
---------------------------------
Attachment: hive-73.12.patch
fixed build-common.xml
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Server Infrastructure
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.10.patch, hive-73.11.patch, hive-73.12.patch, hive-73.2.patch, hive-73.3.txt, hive-73.4.txt, hive-73.5.txt, hive-73.6.patch, hive-73.7.patch, hive-73.8.patch, hive-73.9.patch
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-73) Thrift Server and Client for Hive
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-73:
---------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
HIVE-73. Thrift Server and Client for Hive (Raghu through zshao).
svn revision 723645.
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Server Infrastructure
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.10.patch, hive-73.11.patch, hive-73.12.patch, hive-73.2.patch, hive-73.3.txt, hive-73.4.txt, hive-73.5.txt, hive-73.6.patch, hive-73.7.patch, hive-73.8.patch, hive-73.9.patch
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-73) Thrift Server and Client for Hive
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12651144#action_12651144 ]
Pete Wyckoff commented on HIVE-73:
----------------------------------
some other issues:
1. hardcoded path: +$JAVA_HOME/bin/java -Dcom.sun.management.jmxremote -Djava.library.path=/mnt/hive/production/cluster/lib/native/Linux-amd64-64/ \
2. getStatus and getVersion are already defined in fb303, so why re-defining them?
3. usually fb303 services take advantage of set/getcounter to provide stats on #of various ops and also exceptions. e.g., incrementCounter("queryExceptions");
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.2.patch, hive-73.3.txt, hive-73.4.txt, hive-73.5.txt, hive-73.6.patch, hive-73.7.patch
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-73) Thrift Server and Client for Hive
Posted by "Ashish Thusoo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12651096#action_12651096 ]
Ashish Thusoo commented on HIVE-73:
-----------------------------------
+1
Please file a cleanup JIRA for HiveConf and utility function changes mentioned above and a separate one for multiple statements per session.
Once that is done we can proceed to commit this.
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.2.patch, hive-73.3.txt, hive-73.4.txt, hive-73.5.txt, hive-73.6.patch, hive-73.7.patch
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-73) Thrift Server and Client for Hive
Posted by "Joydeep Sen Sarma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12652567#action_12652567 ]
Joydeep Sen Sarma commented on HIVE-73:
---------------------------------------
+1
do u need to move test.standalone.... to HiveConf.java
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Server Infrastructure
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.10.patch, hive-73.2.patch, hive-73.3.txt, hive-73.4.txt, hive-73.5.txt, hive-73.6.patch, hive-73.7.patch, hive-73.8.patch, hive-73.9.patch
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-73) Thrift Server and Client for Hive
Posted by "Raghotham Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghotham Murthy updated HIVE-73:
---------------------------------
Attachment: hive-73.3.txt
Attaching a smaller diff file without any of the generated code or include files. This should be useful to actually review the code. Note that this file is not the complete patch. I'll upload another file with the full diff after review comments.
In this patch, the schema of the result is now available at the server. It required some hacking of the hive/ql code since there is currently no way to access the schema of the result of a query.
The plan is for the JDBC or other clients to instantiate a DynamicSerDe with the schema string that is returned by the server. The row returned by the server is actually serialized using the MetadataTypedColumnSetSerDe. Pete mentioned that DynamicSerDe can deserialize this row if it is initialized with the right schema and protocol. I'll add that next.
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.2.patch, hive-73.3.txt
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-73) Thrift Server and Client for Hive
Posted by "Raghotham Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghotham Murthy updated HIVE-73:
---------------------------------
Attachment: hive-73.4.txt
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.2.patch, hive-73.3.txt, hive-73.4.txt
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-73) Thrift Server and Client for Hive
Posted by "Joydeep Sen Sarma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12649140#action_12649140 ]
Joydeep Sen Sarma commented on HIVE-73:
---------------------------------------
sounds good.
can u take a look at hive-30 as well? there is some overlap:
- is the thrift server stateful (does it manage sessions)?
- is the thrift api restricted to driver and metastore api? or does it extend to setting up client side session variables and runtime environment (files/jars and other resources)?
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.19.0
> Reporter: Raghotham Murthy
> Fix For: 0.20.0
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-73) Thrift Server and Client for Hive
Posted by "Michi Mutsuzaki (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michi Mutsuzaki updated HIVE-73:
--------------------------------
Attachment: hive-73.8.patch
Some minor chages:
data/conf/hive-site.xml:
- Added description to test.service.standalone.server.
- Changed default to false so that unit test uses embedded mode by default.
service/src/test/org/apache/hadoop/hive/service/TestHiveServer.java:
- Uses getBoolean() to read test.service.standalone.server parameter.
You should be able to apply this patch and run the unit test under service directory without any additional steps.
--Michi
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.2.patch, hive-73.3.txt, hive-73.4.txt, hive-73.5.txt, hive-73.6.patch, hive-73.7.patch, hive-73.8.patch
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-73) Thrift Server and Client for Hive
Posted by "Raghotham Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghotham Murthy updated HIVE-73:
---------------------------------
Attachment: hive-73.10.patch
@joy:
> 1. printInfo vs printError
ideally none of the console outputs other than the data itself should go to stdout. Or we should make "hive -e" default to quiet. We shouldnt have to specify more parameters to just get the data out. I have reverted this change for now. will file another jira for this.
> 2. start_hive_thrift_server.sh has serious issues:
I have removed these scripts. for now made a simple change to bin/hive to start the server if 'service is provided as the first argument. note that this does not daemonize the server.
@pete:
> 1. hardcoded path: +$JAVA_HOME/bin/java -Dcom.sun.management.jmxremote -Djava.library.path=/mnt/hive/production/cluster/lib/native/Linux-amd64-64/ \
No longer hardcoded sice i modified bin/hive
> 2. getStatus and getVersion are already defined in fb303, so why re-defining them?
> 3. usually fb303 services take advantage of set/getcounter to provide stats on #of various ops and also exceptions. e.g., incrementCounter("queryExceptions");
I have gotten rid of fb303 daemonizing stuff until we have a clear way of just using bin/hive for everything. i'd like the client and the server to be two different scripts. but, we could discuss that in a separate jira.
@michi
I have disabled the failing test (should file a separate jira on serde)
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Server Infrastructure
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.10.patch, hive-73.2.patch, hive-73.3.txt, hive-73.4.txt, hive-73.5.txt, hive-73.6.patch, hive-73.7.patch, hive-73.8.patch, hive-73.9.patch
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-73) Thrift Server and Client for Hive
Posted by "Raghotham Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12651118#action_12651118 ]
Raghotham Murthy commented on HIVE-73:
--------------------------------------
I have filed HIVE-80 for the multiple statements feature.
I have already made changes to HiveConf (missed mentioning it in the previous comment). Also, I dont need the utility function since I am now using a pre-existing function in MetaStoreUtils to get the thrift DDL from the deserializer.
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.2.patch, hive-73.3.txt, hive-73.4.txt, hive-73.5.txt, hive-73.6.patch, hive-73.7.patch
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-73) Thrift Server and Client for Hive
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-73:
-------------------------------
Fix Version/s: 0.3.0
(was: 0.6.0)
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Clients, Server Infrastructure
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.3.0
>
> Attachments: hive-73.1.patch, hive-73.10.patch, hive-73.11.patch, hive-73.12.patch, hive-73.2.patch, hive-73.3.txt, hive-73.4.txt, hive-73.5.txt, hive-73.6.patch, hive-73.7.patch, hive-73.8.patch, hive-73.9.patch
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-73) Thrift Server and Client for Hive
Posted by "Michi Mutsuzaki (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michi Mutsuzaki updated HIVE-73:
--------------------------------
Attachment: hive-73.5.txt
This patch implements:
HiveInterface.fetchOne()
HiveInterface.fetchN()
HiveInterface.fetchAll()
Ragho implemented getSchema(). Now we should be able to implement basic jdbc functionality.
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.2.patch, hive-73.3.txt, hive-73.4.txt, hive-73.5.txt
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-73) Thrift Server and Client for Hive
Posted by "Raghotham Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12649208#action_12649208 ]
Raghotham Murthy commented on HIVE-73:
--------------------------------------
@Joy
1. The thrift server is stateful. You can make multiple stateful calls if the transport is not closed between calls.
2. Right now I am just looking at driver and metastore. I guess the client side session variables can be via the 'set' command which can be run as another query. And if the transport is not closed the session variables should persist.
@Zheng
We are planning on supporting multiple fetch calls in the client - fetchOne, fetchN and fetchAll. Since the server is stateful, we should be able to loop through result rows one at a time if required. However, we are still debating if server should also support these methods. We were thinking of just supporting fetchOne and fetchN in the server and then implement fetchAll in the client by calling fetchN underneath. That way the server will not be overloaded in the case of fetchAll.
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.19.0
> Reporter: Raghotham Murthy
> Fix For: 0.20.0
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HIVE-73) Thrift Server and Client for
Hive
Posted by "Raghotham Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12649211#action_12649211 ]
rsm edited comment on HIVE-73 at 11/19/08 3:25 PM:
----------------------------------------------------------------
submitted patch - Look at the tests for usage.
was (Author: rsm):
Look at the tests for usage.
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HIVE-73) Thrift Server and Client for
Hive
Posted by "Raghotham Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12649210#action_12649210 ]
rsm edited comment on HIVE-73 at 11/19/08 3:25 PM:
----------------------------------------------------------------
hive thrift service - attached patch
was (Author: rsm):
hive thrift service
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-73) Thrift Server and Client for Hive
Posted by "Michi Mutsuzaki (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michi Mutsuzaki updated HIVE-73:
--------------------------------
Attachment: hive-73.9.patch
A few more small changes:
- Added a test for dynamic serde. Right now it's failing with this schema: struct result { string _c0}
- Added usefile attribute to junit formatter tag. Set it to false and junit will print output to stdout. Handy when debugging.
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.2.patch, hive-73.3.txt, hive-73.4.txt, hive-73.5.txt, hive-73.6.patch, hive-73.7.patch, hive-73.8.patch, hive-73.9.patch
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-73) Thrift Server and Client for Hive
Posted by "Raghotham Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghotham Murthy updated HIVE-73:
---------------------------------
Attachment: (was: hive-73.6.patch)
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.2.patch, hive-73.3.txt, hive-73.4.txt, hive-73.5.txt
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-73) Thrift Server and Client for Hive
Posted by "Raghotham Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghotham Murthy updated HIVE-73:
---------------------------------
Attachment: hive-73.6.patch
1. there's a lot of weird directories in some of the config/startup scripts. /mnt/vol/hive, ...
removed.
2. why do you need the C++ headers for thrift here? without all of thrift, they cannot compile/run anyway
right now just copied over all thrift specific files from metastore since we were going to get rid of the thrift directories from metastore. ideally, we would expect users to install thrift if they want to compile stuff. so, we can remove it. but will remove it later.
3. for the get type when generating dynamic serde, do you not need to translate types like int to i32?
nice catch. made the change now.
4. many of the exceptions have no info other than the stack trace - i.e., you don't instantiate them with a meaningful string.
fixed.
5. are you sure all the fb303 functions you are using are in the current public release of thrift?
yes. need to install the python modules in thrift/lib/py and thrift/contrib/fb303/py. the fb303_simple_mgmt script has to be copied over from thrift/contrib/fb303/scripts/.
Attaching a new patch.
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.2.patch, hive-73.3.txt, hive-73.4.txt, hive-73.5.txt, hive-73.6.patch
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-73) Thrift Server and Client for Hive
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12649204#action_12649204 ]
Zheng Shao commented on HIVE-73:
--------------------------------
If the query is a temporary query that returns results, will we return one row in one thrift call? (I guess thrift does not support data streaming).
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.19.0
> Reporter: Raghotham Murthy
> Fix For: 0.20.0
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HIVE-73) Thrift Server and Client for
Hive
Posted by "Michi Mutsuzaki (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12652297#action_12652297 ]
michim edited comment on HIVE-73 at 12/1/08 10:55 PM:
---------------------------------------------------------------
> @michi
> I have disabled the failing test (should file a separate jira on serde)
I filed a bug on this (HIVE-99).
was (Author: michim):
> @michi
> I have disabled the failing test (should file a separate jira on serde)
I filed a bug on this.
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Server Infrastructure
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.10.patch, hive-73.2.patch, hive-73.3.txt, hive-73.4.txt, hive-73.5.txt, hive-73.6.patch, hive-73.7.patch, hive-73.8.patch, hive-73.9.patch
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HIVE-73) Thrift Server and Client for Hive
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur reassigned HIVE-73:
------------------------------------
Assignee: Raghotham Murthy
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-73) Thrift Server and Client for Hive
Posted by "Raghotham Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghotham Murthy updated HIVE-73:
---------------------------------
Attachment: hive-73.1.patch
hive thrift service
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-73) Thrift Server and Client for Hive
Posted by "Raghotham Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653085#action_12653085 ]
Raghotham Murthy commented on HIVE-73:
--------------------------------------
@joy, we dont need to move test.standalone to HiveConf.java. The latest patch adds standalone to build-common.xml so that ant -Dstandalone=true test will result in using the standalone server.
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Server Infrastructure
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.10.patch, hive-73.11.patch, hive-73.2.patch, hive-73.3.txt, hive-73.4.txt, hive-73.5.txt, hive-73.6.patch, hive-73.7.patch, hive-73.8.patch, hive-73.9.patch
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-73) Thrift Server and Client for Hive
Posted by "Pete Wyckoff (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12650306#action_12650306 ]
Pete Wyckoff commented on HIVE-73:
----------------------------------
overall looks good, but some comments on the code:
1. there's a lot of weird directories in some of the config/startup scripts. /mnt/vol/hive, ...
2. why do you need the C++ headers for thrift here? without all of thrift, they cannot compile/run anyway
3. for the get type when generating dynamic serde, do you not need to translate types like int to i32?
4. many of the exceptions have no info other than the stack trace - i.e., you don't instantiate them with a meaningful string.
5. are you sure all the fb303 functions you are using are in the current public release of thrift?
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.2.patch, hive-73.3.txt, hive-73.4.txt, hive-73.5.txt
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-73) Thrift Server and Client for Hive
Posted by "Raghotham Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghotham Murthy updated HIVE-73:
---------------------------------
Attachment: (was: hive-73.2.patch)
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-73) Thrift Server and Client for Hive
Posted by "Raghotham Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghotham Murthy updated HIVE-73:
---------------------------------
Attachment: hive-73.6.patch
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.2.patch, hive-73.3.txt, hive-73.4.txt, hive-73.5.txt, hive-73.6.patch
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-73) Thrift Server and Client for Hive
Posted by "Raghotham Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghotham Murthy updated HIVE-73:
---------------------------------
Attachment: (was: hive-73.10.patch)
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Server Infrastructure
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.10.patch, hive-73.2.patch, hive-73.3.txt, hive-73.4.txt, hive-73.5.txt, hive-73.6.patch, hive-73.7.patch, hive-73.8.patch, hive-73.9.patch
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-73) Thrift Server and Client for Hive
Posted by "Raghotham Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghotham Murthy updated HIVE-73:
---------------------------------
Attachment: hive-73.2.patch
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.2.patch
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-73) Thrift Server and Client for Hive
Posted by "Raghotham Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghotham Murthy updated HIVE-73:
---------------------------------
Affects Version/s: (was: 0.19.0)
0.20.0
Status: Patch Available (was: Open)
Look at the tests for usage.
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-73) Thrift Server and Client for Hive
Posted by "Raghotham Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghotham Murthy updated HIVE-73:
---------------------------------
Attachment: hive-73.11.patch
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Server Infrastructure
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.10.patch, hive-73.11.patch, hive-73.2.patch, hive-73.3.txt, hive-73.4.txt, hive-73.5.txt, hive-73.6.patch, hive-73.7.patch, hive-73.8.patch, hive-73.9.patch
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HIVE-73) Thrift Server and Client for
Hive
Posted by "Michi Mutsuzaki (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12650110#action_12650110 ]
michim edited comment on HIVE-73 at 11/23/08 9:16 PM:
---------------------------------------------------------------
This patch (hive-73.5.txt) implements:
HiveInterface.fetchOne()
HiveInterface.fetchN()
HiveInterface.fetchAll()
Ragho implemented getSchema(). Now we should be able to implement basic jdbc functionality.
was (Author: michim):
This patch implements:
HiveInterface.fetchOne()
HiveInterface.fetchN()
HiveInterface.fetchAll()
Ragho implemented getSchema(). Now we should be able to implement basic jdbc functionality.
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.2.patch, hive-73.3.txt, hive-73.4.txt, hive-73.5.txt
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-73) Thrift Server and Client for Hive
Posted by "Raghotham Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghotham Murthy updated HIVE-73:
---------------------------------
Attachment: (was: hive-73.4.txt)
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.2.patch, hive-73.3.txt, hive-73.4.txt
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-73) Thrift Server and Client for Hive
Posted by "Joydeep Sen Sarma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12651140#action_12651140 ]
Joydeep Sen Sarma commented on HIVE-73:
---------------------------------------
start_hive_thrift_server.sh has serious issues. i can't understand why this is different from bin/hive. everytime we fork this logic - we miss out on stuff. look at stuff that's missing (wrt bin/hive):
- auxlibs?
- hardcoding lib/native location - this is just not the way to do this - when we go through bin/hive (which goes through hadoop) - this gets set to the directory corresponding to java version and machine arch
why is it not possible to 'case' bin/hive to launch different classes based on some input parameters (as was discussed in hive-30 - which hopefully Edward is also working on?)
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.2.patch, hive-73.3.txt, hive-73.4.txt, hive-73.5.txt, hive-73.6.patch, hive-73.7.patch
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-73) Thrift Server and Client for Hive
Posted by "Raghotham Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghotham Murthy updated HIVE-73:
---------------------------------
Attachment: hive-73.4.txt
attaching new (readable) diff without generated code/include files. In this patch, added test code to read data returned by server using DynamicSerDe.
Discovered a couple of bugs (in other parts of hive) while testing this code.
1. select * queries dont generate schema of the result
2. ColumnInfo (type) of the result columns are not generated correctly. for example, create table test(num int). Then the ColumnInfo for the result of query 'select num from test where num > 10' (added num > 10 to get past bug 1) has type 'string'.
I am not sure these two bugs need to be fixed as part of this jira.
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.2.patch, hive-73.3.txt, hive-73.4.txt
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-73) Thrift Server and Client for Hive
Posted by "Michi Mutsuzaki (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12650415#action_12650415 ]
Michi Mutsuzaki commented on HIVE-73:
-------------------------------------
Hi Pete, thanks for your feedback.
4. many of the exceptions have no info other than the stack trace - i.e., you don't instantiate them with a meaningful string.
Yes, I'll fix this.
You need to check with Ragho on 1,2,3, and 5.
--Michi
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.2.patch, hive-73.3.txt, hive-73.4.txt, hive-73.5.txt
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-73) Thrift Server and Client for Hive
Posted by "Joydeep Sen Sarma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12652246#action_12652246 ]
Joydeep Sen Sarma commented on HIVE-73:
---------------------------------------
regarding the silent mode - we simply followed mysql and sqlplus precedent - in both of these a 'silent' mode has to be explicitly specified to get only data (even in the equivalent of the '-e' mode). not sure there's any pressing reason to change as long as things are well documented (which probably they aren't - but that's a separate issue).
bin/hive - Edward also made similar changes i think (haven't seen). some coordination with hive-30 may be needed ..
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Server Infrastructure
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.10.patch, hive-73.2.patch, hive-73.3.txt, hive-73.4.txt, hive-73.5.txt, hive-73.6.patch, hive-73.7.patch, hive-73.8.patch, hive-73.9.patch
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-73) Thrift Server and Client for Hive
Posted by "Ashish Thusoo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashish Thusoo updated HIVE-73:
------------------------------
Component/s: Server Infrastructure
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Server Infrastructure
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.2.patch, hive-73.3.txt, hive-73.4.txt, hive-73.5.txt, hive-73.6.patch, hive-73.7.patch, hive-73.8.patch, hive-73.9.patch
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-73) Thrift Server and Client for Hive
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-73:
-------------------------------
Component/s: Clients
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Clients, Server Infrastructure
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.3.0
>
> Attachments: hive-73.1.patch, hive-73.10.patch, hive-73.11.patch, hive-73.12.patch, hive-73.2.patch, hive-73.3.txt, hive-73.4.txt, hive-73.5.txt, hive-73.6.patch, hive-73.7.patch, hive-73.8.patch, hive-73.9.patch
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-73) Thrift Server and Client for Hive
Posted by "Joydeep Sen Sarma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12650839#action_12650839 ]
Joydeep Sen Sarma commented on HIVE-73:
---------------------------------------
should have hive-77 fixed in a day or two unless something unexpected comes up
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.2.patch, hive-73.3.txt, hive-73.4.txt, hive-73.5.txt, hive-73.6.patch
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-73) Thrift Server and Client for Hive
Posted by "Raghotham Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghotham Murthy updated HIVE-73:
---------------------------------
Attachment: hive-73.10.patch
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Server Infrastructure
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.10.patch, hive-73.2.patch, hive-73.3.txt, hive-73.4.txt, hive-73.5.txt, hive-73.6.patch, hive-73.7.patch, hive-73.8.patch, hive-73.9.patch
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-73) Thrift Server and Client for Hive
Posted by "Joydeep Sen Sarma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12651135#action_12651135 ]
Joydeep Sen Sarma commented on HIVE-73:
---------------------------------------
quick comment:
- console.printInfo("OK");
+ console.printError("OK");
not on board with this - perhaps the stdout was causing grief to the final results (not clear to me)? anyway - if this is the case - please mark the session as silent (see SessionState.java) - instead of changing the print directives.
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.2.patch, hive-73.3.txt, hive-73.4.txt, hive-73.5.txt, hive-73.6.patch, hive-73.7.patch
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-73) Thrift Server and Client for Hive
Posted by "Ashish Thusoo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-73?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12650812#action_12650812 ]
Ashish Thusoo commented on HIVE-73:
-----------------------------------
Michi.. I think with the following code review encapsulates many of the discussions.. once these are addressed we can move to commit this in...
This is from a combined code review with Namit, Ashish and Raghu...
Please use the functions in MetastoreUtils.java to create the thrift DDL from tableDesc stored in fetchTask via that loadFileWork.
Rest of the comments are inlined.
Inline Comments
service/src/java/org/apache/hadoop/hive/service/HiveServer.java:195 We should avoid the copy here. It is better to accumulate only the necessary number of rows and then pass that list out.
service/src/java/org/apache/hadoop/hive/service/HiveServer.java:134 Can you make this a utility function.
service/src/java/org/apache/hadoop/hive/service/HiveServer.java:71 Can you move this to HiveConf.
service/src/java/org/apache/hadoop/hive/service/HiveServer.java:85 Since JDBC support multiple statements per session, we need to have a mapping from session to statements and then from statement to driver.
service/src/java/org/apache/hadoop/hive/service/HiveServer.java:220 Can you add javadocs.
service/src/java/org/apache/hadoop/hive/service/HiveServer.java:221 These are not in the thrift interface?
service/src/java/org/apache/hadoop/hive/service/HiveServer.java:87 Session state is not thread safe (it is a singleton), so this is blocked on the thread safety JIRA
service/src/java/org/apache/hadoop/hive/service/HiveClient.java:41 javadocs.
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:1869 Should use tableDesc here.
ql/src/java/org/apache/hadoop/hive/ql/Driver.java:73 javadocs..
ql/src/java/org/apache/hadoop/hive/ql/Driver.java:85 javadocs..
ql/src/java/org/apache/hadoop/hive/ql/Driver.java:73 javadocs
> Thrift Server and Client for Hive
> ---------------------------------
>
> Key: HIVE-73
> URL: https://issues.apache.org/jira/browse/HIVE-73
> Project: Hadoop Hive
> Issue Type: New Feature
> Affects Versions: 0.20.0
> Reporter: Raghotham Murthy
> Assignee: Raghotham Murthy
> Fix For: 0.20.0
>
> Attachments: hive-73.1.patch, hive-73.2.patch, hive-73.3.txt, hive-73.4.txt, hive-73.5.txt, hive-73.6.patch
>
>
> Currently the hive cli directly calls the driver code. We need to be able to run a stand alone hive server that multiple clients can connect to. The hive server will allow clients to run queries as well as make meta data calls (by inheriting from the thrift metastore server)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.