You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Sohan Jain (JIRA)" <ji...@apache.org> on 2011/05/27 21:56:47 UTC
[jira] [Created] (HIVE-2188) Add multi_get_table function in Hive
Metastore
Add multi_get_table function in Hive Metastore
----------------------------------------------
Key: HIVE-2188
URL: https://issues.apache.org/jira/browse/HIVE-2188
Project: Hive
Issue Type: New Feature
Components: Metastore
Reporter: Sohan Jain
Assignee: Sohan Jain
Priority: Minor
This function would get multiple tables from the hive metastore as opposed to just one at a time, saving round trip time to the metastore.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2188) Add multi_get_table function in Hive
Metastore
Posted by "Sohan Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sohan Jain updated HIVE-2188:
-----------------------------
Attachment: HIVE-2188.3.patch
> Add multi_get_table function in Hive Metastore
> ----------------------------------------------
>
> Key: HIVE-2188
> URL: https://issues.apache.org/jira/browse/HIVE-2188
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sohan Jain
> Assignee: Sohan Jain
> Priority: Minor
> Attachments: HIVE-2188.1.patch, HIVE-2188.3.patch
>
>
> This function would get multiple tables from the hive metastore as opposed to just one at a time, saving round trip time to the metastore.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2188) Add multi_get_table function in Hive
Metastore
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13045229#comment-13045229 ]
jiraposter@reviews.apache.org commented on HIVE-2188:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/831/
-----------------------------------------------------------
(Updated 2011-06-07 01:11:44.582629)
Review request for hive, Paul Yang and Ashutosh Chauhan.
Changes
-------
- included Carl's suggestions.
- get_table_objects_by_name retrieves only the tables that it can, and does not fail out if it cannot retrieve a table.
- removed the ordering of tables returned, since there is no guarantee every requested table is returned
- modified unit tests
Summary
-------
Created a function "multi_get_table" that retrieves multiple tables on one trip to the hive metastore, saving round trip time.
This addresses bug HIVE-2188.
https://issues.apache.org/jira/browse/HIVE-2188
Diffs (updated)
-----
trunk/metastore/if/hive_metastore.thrift 1130342
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1130342
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1130342
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1130342
trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java 1130342
Diff: https://reviews.apache.org/r/831/diff
Testing
-------
Added a test case to testMetasore() in TestHiveServer. Also tested for speed improvements in a client session.
Thanks,
Sohan
> Add multi_get_table function in Hive Metastore
> ----------------------------------------------
>
> Key: HIVE-2188
> URL: https://issues.apache.org/jira/browse/HIVE-2188
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sohan Jain
> Assignee: Sohan Jain
> Priority: Minor
> Attachments: HIVE-2188.1.patch, HIVE-2188.3.patch
>
>
> This function would get multiple tables from the hive metastore as opposed to just one at a time, saving round trip time to the metastore.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2188) Add get_table_objects_by_name() to
Hive MetaStore
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13047150#comment-13047150 ]
Hudson commented on HIVE-2188:
------------------------------
Integrated in Hive-trunk-h0.21 #772 (See [https://builds.apache.org/job/Hive-trunk-h0.21/772/])
HIVE-2188. Add get_table_objects_by_name() to Hive MetaStore (Sohan Jain via cws)
cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1134183
Files :
* /hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java
* /hive/trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp
* /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
* /hive/trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h
* /hive/trunk/metastore/if/hive_metastore.thrift
* /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote
* /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java
* /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
* /hive/trunk/metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp
* /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py
* /hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php
* /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java
* /hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java
* /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
* /hive/trunk/metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb
> Add get_table_objects_by_name() to Hive MetaStore
> -------------------------------------------------
>
> Key: HIVE-2188
> URL: https://issues.apache.org/jira/browse/HIVE-2188
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sohan Jain
> Assignee: Sohan Jain
> Priority: Minor
> Fix For: 0.8.0
>
> Attachments: HIVE-2188.1.patch, HIVE-2188.3.patch
>
>
> This function would get multiple tables from the hive metastore as opposed to just one at a time, saving round trip time to the metastore.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2188) Add get_table_objects_by_name() to
Hive MetaStore
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-2188:
---------------------------------
Summary: Add get_table_objects_by_name() to Hive MetaStore (was: Add multi_get_table function in Hive Metastore)
> Add get_table_objects_by_name() to Hive MetaStore
> -------------------------------------------------
>
> Key: HIVE-2188
> URL: https://issues.apache.org/jira/browse/HIVE-2188
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sohan Jain
> Assignee: Sohan Jain
> Priority: Minor
> Attachments: HIVE-2188.1.patch, HIVE-2188.3.patch
>
>
> This function would get multiple tables from the hive metastore as opposed to just one at a time, saving round trip time to the metastore.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2188) Add multi_get_table function in Hive
Metastore
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13045610#comment-13045610 ]
jiraposter@reviews.apache.org commented on HIVE-2188:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/831/#review778
-----------------------------------------------------------
trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java
<https://reviews.apache.org/r/831/#comment1676>
Sorry I missed this before, but you need to add a getTableObjectsByName() method to IMetaStoreClient and HiveMetaStoreClient, and use that method here instead of calling client.client.get_table_objects_by_name(). Everything else looks good so, so you can either update this patch, or else open another ticket to address this issue. Please let me know what you want to do. Thanks.
- Carl
On 2011-06-07 01:11:44, Sohan Jain wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/831/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-06-07 01:11:44)
bq.
bq.
bq. Review request for hive, Paul Yang and Ashutosh Chauhan.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Created a function "multi_get_table" that retrieves multiple tables on one trip to the hive metastore, saving round trip time.
bq.
bq.
bq. This addresses bug HIVE-2188.
bq. https://issues.apache.org/jira/browse/HIVE-2188
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. trunk/metastore/if/hive_metastore.thrift 1130342
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1130342
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1130342
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1130342
bq. trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java 1130342
bq.
bq. Diff: https://reviews.apache.org/r/831/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Added a test case to testMetasore() in TestHiveServer. Also tested for speed improvements in a client session.
bq.
bq.
bq. Thanks,
bq.
bq. Sohan
bq.
bq.
> Add multi_get_table function in Hive Metastore
> ----------------------------------------------
>
> Key: HIVE-2188
> URL: https://issues.apache.org/jira/browse/HIVE-2188
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sohan Jain
> Assignee: Sohan Jain
> Priority: Minor
> Attachments: HIVE-2188.1.patch, HIVE-2188.3.patch
>
>
> This function would get multiple tables from the hive metastore as opposed to just one at a time, saving round trip time to the metastore.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-2188) Add get_table_objects_by_name() to
Hive MetaStore
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach resolved HIVE-2188.
----------------------------------
Resolution: Fixed
Fix Version/s: 0.8.0
Hadoop Flags: [Reviewed]
Committed to trunk. Thanks Sohan!
> Add get_table_objects_by_name() to Hive MetaStore
> -------------------------------------------------
>
> Key: HIVE-2188
> URL: https://issues.apache.org/jira/browse/HIVE-2188
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sohan Jain
> Assignee: Sohan Jain
> Priority: Minor
> Fix For: 0.8.0
>
> Attachments: HIVE-2188.1.patch, HIVE-2188.3.patch
>
>
> This function would get multiple tables from the hive metastore as opposed to just one at a time, saving round trip time to the metastore.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2188) Add multi_get_table function in Hive
Metastore
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13045111#comment-13045111 ]
jiraposter@reviews.apache.org commented on HIVE-2188:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/831/
-----------------------------------------------------------
(Updated 2011-06-06 21:09:54.167933)
Review request for hive, Paul Yang and Ashutosh Chauhan.
Changes
-------
- Changed function name to get_multi_table
- Fixed returned ordering of requested tables
- Moved and added additional test cases
Summary
-------
Created a function "multi_get_table" that retrieves multiple tables on one trip to the hive metastore, saving round trip time.
This addresses bug HIVE-2188.
https://issues.apache.org/jira/browse/HIVE-2188
Diffs (updated)
-----
trunk/metastore/if/hive_metastore.thrift 1130342
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1130342
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1130342
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1130342
trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java 1130342
Diff: https://reviews.apache.org/r/831/diff
Testing
-------
Added a test case to testMetasore() in TestHiveServer. Also tested for speed improvements in a client session.
Thanks,
Sohan
> Add multi_get_table function in Hive Metastore
> ----------------------------------------------
>
> Key: HIVE-2188
> URL: https://issues.apache.org/jira/browse/HIVE-2188
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sohan Jain
> Assignee: Sohan Jain
> Priority: Minor
> Attachments: HIVE-2188.1.patch
>
>
> This function would get multiple tables from the hive metastore as opposed to just one at a time, saving round trip time to the metastore.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2188) Add multi_get_table function in Hive
Metastore
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13045121#comment-13045121 ]
jiraposter@reviews.apache.org commented on HIVE-2188:
-----------------------------------------------------
bq. On 2011-06-03 14:04:02, Ashutosh Chauhan wrote:
bq. > trunk/metastore/if/hive_metastore.thrift, line 249
bq. > <https://reviews.apache.org/r/831/diff/1/?file=20144#file20144line249>
bq. >
bq. > How about calling it get_multi_table instead? multi_get_table sounds little confusing to me.
Ok, get_multi_table definitely sounds better.
bq. On 2011-06-03 14:04:02, Ashutosh Chauhan wrote:
bq. > trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, line 521
bq. > <https://reviews.apache.org/r/831/diff/1/?file=20145#file20145line521>
bq. >
bq. > You can write this more concisely using commons-lang utility method as: StringUtils.join(tbls,',');
Thanks for the tip; I changed this.
bq. On 2011-06-03 14:04:02, Ashutosh Chauhan wrote:
bq. > trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, line 1182
bq. > <https://reviews.apache.org/r/831/diff/1/?file=20145#file20145line1182>
bq. >
bq. > You can get rid of tables.get(i) == null check that will never be true.
I cleaned up the checks for seeing if the db had all the requested tables. Also, I realized that I did not guarantee returning tables in the same order as those requested, so I fixed that up too.
bq. On 2011-06-03 14:04:02, Ashutosh Chauhan wrote:
bq. > trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, line 1195
bq. > <https://reviews.apache.org/r/831/diff/1/?file=20145#file20145line1195>
bq. >
bq. > Instead of throwing RuntimeException, create MetaException and throw that.
I changed it to throw a MetaException instead. However, the other metastore operations (e.g., get_table, append_partition) throw RuntimException, which is what I based get_multi_table off.
bq. On 2011-06-03 14:04:02, Ashutosh Chauhan wrote:
bq. > trunk/service/src/test/org/apache/hadoop/hive/service/TestHiveServer.java, line 221
bq. > <https://reviews.apache.org/r/831/diff/1/?file=20148#file20148line221>
bq. >
bq. > This test really belongs in the TestMetastore or some such in metastore dir not in HiveServer.
Thanks for the pointer and for the feedback.
- Sohan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/831/#review753
-----------------------------------------------------------
On 2011-06-06 21:09:54, Sohan Jain wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/831/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-06-06 21:09:54)
bq.
bq.
bq. Review request for hive, Paul Yang and Ashutosh Chauhan.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Created a function "multi_get_table" that retrieves multiple tables on one trip to the hive metastore, saving round trip time.
bq.
bq.
bq. This addresses bug HIVE-2188.
bq. https://issues.apache.org/jira/browse/HIVE-2188
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. trunk/metastore/if/hive_metastore.thrift 1130342
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1130342
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1130342
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1130342
bq. trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java 1130342
bq.
bq. Diff: https://reviews.apache.org/r/831/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Added a test case to testMetasore() in TestHiveServer. Also tested for speed improvements in a client session.
bq.
bq.
bq. Thanks,
bq.
bq. Sohan
bq.
bq.
> Add multi_get_table function in Hive Metastore
> ----------------------------------------------
>
> Key: HIVE-2188
> URL: https://issues.apache.org/jira/browse/HIVE-2188
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sohan Jain
> Assignee: Sohan Jain
> Priority: Minor
> Attachments: HIVE-2188.1.patch
>
>
> This function would get multiple tables from the hive metastore as opposed to just one at a time, saving round trip time to the metastore.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2188) Add multi_get_table function in Hive
Metastore
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-2188:
---------------------------------
Status: Open (was: Patch Available)
> Add multi_get_table function in Hive Metastore
> ----------------------------------------------
>
> Key: HIVE-2188
> URL: https://issues.apache.org/jira/browse/HIVE-2188
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sohan Jain
> Assignee: Sohan Jain
> Priority: Minor
> Attachments: HIVE-2188.1.patch, HIVE-2188.3.patch
>
>
> This function would get multiple tables from the hive metastore as opposed to just one at a time, saving round trip time to the metastore.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2188) Add multi_get_table function in Hive
Metastore
Posted by "Sohan Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13042532#comment-13042532 ]
Sohan Jain commented on HIVE-2188:
----------------------------------
RB: https://reviews.apache.org/r/831/
Thanks for looking, Ashutosh. I'm not sure how to translate the speedup to hive queries, but a plausible use case is when you are working with a Hive client. Suppose you have N tables that you know share a set of keys, and you need to get the values of those keys for each table somewhat quickly to populate a UI. Previously, you would have to call get_table on each table, which requires N trips to the metastore, but now you can retrieve all tables with just one trip to the metastore, saving time on the overhead of going to the metastore.
> Add multi_get_table function in Hive Metastore
> ----------------------------------------------
>
> Key: HIVE-2188
> URL: https://issues.apache.org/jira/browse/HIVE-2188
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sohan Jain
> Assignee: Sohan Jain
> Priority: Minor
> Attachments: HIVE-2188.1.patch
>
>
> This function would get multiple tables from the hive metastore as opposed to just one at a time, saving round trip time to the metastore.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2188) Add multi_get_table function in Hive
Metastore
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13045128#comment-13045128 ]
jiraposter@reviews.apache.org commented on HIVE-2188:
-----------------------------------------------------
bq. On 2011-06-03 14:04:02, Ashutosh Chauhan wrote:
bq. > trunk/metastore/if/hive_metastore.thrift, line 249
bq. > <https://reviews.apache.org/r/831/diff/1/?file=20144#file20144line249>
bq. >
bq. > How about calling it get_multi_table instead? multi_get_table sounds little confusing to me.
bq.
bq. Sohan Jain wrote:
bq. Ok, get_multi_table definitely sounds better.
"get_tables" is probably the ideal name, but unfortunately that's already taken by a function that should really be called "get_table_names_by_pattern". In order to avoid this problem going forward, what do you think of changing the name to "get_table_objects"?
- Carl
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/831/#review753
-----------------------------------------------------------
On 2011-06-06 21:09:54, Sohan Jain wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/831/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-06-06 21:09:54)
bq.
bq.
bq. Review request for hive, Paul Yang and Ashutosh Chauhan.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Created a function "multi_get_table" that retrieves multiple tables on one trip to the hive metastore, saving round trip time.
bq.
bq.
bq. This addresses bug HIVE-2188.
bq. https://issues.apache.org/jira/browse/HIVE-2188
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. trunk/metastore/if/hive_metastore.thrift 1130342
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1130342
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1130342
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1130342
bq. trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java 1130342
bq.
bq. Diff: https://reviews.apache.org/r/831/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Added a test case to testMetasore() in TestHiveServer. Also tested for speed improvements in a client session.
bq.
bq.
bq. Thanks,
bq.
bq. Sohan
bq.
bq.
> Add multi_get_table function in Hive Metastore
> ----------------------------------------------
>
> Key: HIVE-2188
> URL: https://issues.apache.org/jira/browse/HIVE-2188
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sohan Jain
> Assignee: Sohan Jain
> Priority: Minor
> Attachments: HIVE-2188.1.patch, HIVE-2188.3.patch
>
>
> This function would get multiple tables from the hive metastore as opposed to just one at a time, saving round trip time to the metastore.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2188) Add multi_get_table function in Hive
Metastore
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13045174#comment-13045174 ]
jiraposter@reviews.apache.org commented on HIVE-2188:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/831/#review768
-----------------------------------------------------------
trunk/metastore/if/hive_metastore.thrift
<https://reviews.apache.org/r/831/#comment1650>
Please consider changing this to "get_table_objects_by_name"
This should also throw InvalidObjectException and UnknownDBException.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
<https://reviews.apache.org/r/831/#comment1648>
Maybe change this to "get_table_objects_by_name" in order to disambiguate from cases where we're returning only tables names, or applying a filter condition, etc. etc.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
<https://reviews.apache.org/r/831/#comment1647>
This should probably be InvalidOperationException instead of NoSuchObjectException.
It might also be good to validate the dbname input parameter at this step, e.g. make sure it's not null and not an empty string.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
<https://reviews.apache.org/r/831/#comment1646>
Failing the entire operation if a single table in the input list is not defined seems like a bad idea since we're throwing away work that will have to be repeated on the next call. Furthermore, the exception doesn't contain any information about which table(s) are not defined, so the client will have to fetch a table list again and use this to construct the list of input tables for the next get_multi_table() call. In the meantime it's possible that someone will drop a table in the list, which will invalidate the next call.
I think it would be better to modify the contract to state that if a table on the input list is not found in the metastore, then the table definition will not be included in the result. This means that the function will return an empty list if none of the tables in the input list are found in the metastore.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
<https://reviews.apache.org/r/831/#comment1645>
e.toString() actually returns a little more information than e.getMessage().
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
<https://reviews.apache.org/r/831/#comment1649>
Change to getTableObjectsByName?
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
<https://reviews.apache.org/r/831/#comment1643>
It would be good to first check if the DB exists, and throw UnknownDBException if it's not found.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
<https://reviews.apache.org/r/831/#comment1644>
Only some callers will care about having this ordering property satisfied, so instead of penalizing every caller with this performance hit, maybe it would be better to let the caller take care of this?
- Carl
On 2011-06-06 21:09:54, Sohan Jain wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/831/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-06-06 21:09:54)
bq.
bq.
bq. Review request for hive, Paul Yang and Ashutosh Chauhan.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Created a function "multi_get_table" that retrieves multiple tables on one trip to the hive metastore, saving round trip time.
bq.
bq.
bq. This addresses bug HIVE-2188.
bq. https://issues.apache.org/jira/browse/HIVE-2188
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. trunk/metastore/if/hive_metastore.thrift 1130342
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1130342
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1130342
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1130342
bq. trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java 1130342
bq.
bq. Diff: https://reviews.apache.org/r/831/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Added a test case to testMetasore() in TestHiveServer. Also tested for speed improvements in a client session.
bq.
bq.
bq. Thanks,
bq.
bq. Sohan
bq.
bq.
> Add multi_get_table function in Hive Metastore
> ----------------------------------------------
>
> Key: HIVE-2188
> URL: https://issues.apache.org/jira/browse/HIVE-2188
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sohan Jain
> Assignee: Sohan Jain
> Priority: Minor
> Attachments: HIVE-2188.1.patch, HIVE-2188.3.patch
>
>
> This function would get multiple tables from the hive metastore as opposed to just one at a time, saving round trip time to the metastore.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2188) Add get_table_objects_by_name() to
Hive MetaStore
Posted by "Sohan Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13046983#comment-13046983 ]
Sohan Jain commented on HIVE-2188:
----------------------------------
Thank you, Carl.
> Add get_table_objects_by_name() to Hive MetaStore
> -------------------------------------------------
>
> Key: HIVE-2188
> URL: https://issues.apache.org/jira/browse/HIVE-2188
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sohan Jain
> Assignee: Sohan Jain
> Priority: Minor
> Fix For: 0.8.0
>
> Attachments: HIVE-2188.1.patch, HIVE-2188.3.patch
>
>
> This function would get multiple tables from the hive metastore as opposed to just one at a time, saving round trip time to the metastore.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2188) Add multi_get_table function in Hive
Metastore
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13045759#comment-13045759 ]
jiraposter@reviews.apache.org commented on HIVE-2188:
-----------------------------------------------------
bq. On 2011-06-07 20:09:34, Carl Steinbach wrote:
bq. > trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java, line 748
bq. > <https://reviews.apache.org/r/831/diff/3/?file=20654#file20654line748>
bq. >
bq. > Sorry I missed this before, but you need to add a getTableObjectsByName() method to IMetaStoreClient and HiveMetaStoreClient, and use that method here instead of calling client.client.get_table_objects_by_name(). Everything else looks good so, so you can either update this patch, or else open another ticket to address this issue. Please let me know what you want to do. Thanks.
Hi Carl, I have updated the patch here. Thanks for your help.
- Sohan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/831/#review778
-----------------------------------------------------------
On 2011-06-07 01:11:44, Sohan Jain wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/831/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-06-07 01:11:44)
bq.
bq.
bq. Review request for hive, Paul Yang and Ashutosh Chauhan.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Created a function "multi_get_table" that retrieves multiple tables on one trip to the hive metastore, saving round trip time.
bq.
bq.
bq. This addresses bug HIVE-2188.
bq. https://issues.apache.org/jira/browse/HIVE-2188
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. trunk/metastore/if/hive_metastore.thrift 1130342
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1130342
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1130342
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1130342
bq. trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java 1130342
bq.
bq. Diff: https://reviews.apache.org/r/831/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Added a test case to testMetasore() in TestHiveServer. Also tested for speed improvements in a client session.
bq.
bq.
bq. Thanks,
bq.
bq. Sohan
bq.
bq.
> Add multi_get_table function in Hive Metastore
> ----------------------------------------------
>
> Key: HIVE-2188
> URL: https://issues.apache.org/jira/browse/HIVE-2188
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sohan Jain
> Assignee: Sohan Jain
> Priority: Minor
> Attachments: HIVE-2188.1.patch, HIVE-2188.3.patch
>
>
> This function would get multiple tables from the hive metastore as opposed to just one at a time, saving round trip time to the metastore.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2188) Add multi_get_table function in Hive
Metastore
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13043362#comment-13043362 ]
jiraposter@reviews.apache.org commented on HIVE-2188:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/831/#review753
-----------------------------------------------------------
trunk/metastore/if/hive_metastore.thrift
<https://reviews.apache.org/r/831/#comment1571>
How about calling it get_multi_table instead? multi_get_table sounds little confusing to me.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
<https://reviews.apache.org/r/831/#comment1572>
You can write this more concisely using commons-lang utility method as: StringUtils.join(tbls,',');
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
<https://reviews.apache.org/r/831/#comment1576>
You can get rid of tables.get(i) == null check that will never be true.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
<https://reviews.apache.org/r/831/#comment1573>
Instead of throwing RuntimeException, create MetaException and throw that.
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java
<https://reviews.apache.org/r/831/#comment1574>
Please add javadocs for new methods introduced in interface. Also see my first comment for name.
trunk/service/src/test/org/apache/hadoop/hive/service/TestHiveServer.java
<https://reviews.apache.org/r/831/#comment1575>
This test really belongs in the TestMetastore or some such in metastore dir not in HiveServer.
- Ashutosh
On 2011-06-02 23:01:00, Sohan Jain wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/831/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-06-02 23:01:00)
bq.
bq.
bq. Review request for hive, Paul Yang and Ashutosh Chauhan.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Created a function "multi_get_table" that retrieves multiple tables on one trip to the hive metastore, saving round trip time.
bq.
bq.
bq. This addresses bug HIVE-2188.
bq. https://issues.apache.org/jira/browse/HIVE-2188
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. trunk/metastore/if/hive_metastore.thrift 1130342
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1130342
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1130342
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1130342
bq. trunk/service/src/test/org/apache/hadoop/hive/service/TestHiveServer.java 1130342
bq.
bq. Diff: https://reviews.apache.org/r/831/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Added a test case to testMetasore() in TestHiveServer. Also tested for speed improvements in a client session.
bq.
bq.
bq. Thanks,
bq.
bq. Sohan
bq.
bq.
> Add multi_get_table function in Hive Metastore
> ----------------------------------------------
>
> Key: HIVE-2188
> URL: https://issues.apache.org/jira/browse/HIVE-2188
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sohan Jain
> Assignee: Sohan Jain
> Priority: Minor
> Attachments: HIVE-2188.1.patch
>
>
> This function would get multiple tables from the hive metastore as opposed to just one at a time, saving round trip time to the metastore.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2188) Add multi_get_table function in Hive
Metastore
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13042312#comment-13042312 ]
Ashutosh Chauhan commented on HIVE-2188:
----------------------------------------
Hey Sohan, Useful feature.
1. Can you explain a bit in form of hive queries where this speedup will be useful?
2. Can you please create a review board for it?
3. Also, just tease out generated code and just post your changes on RB, that will make it easier to review.
> Add multi_get_table function in Hive Metastore
> ----------------------------------------------
>
> Key: HIVE-2188
> URL: https://issues.apache.org/jira/browse/HIVE-2188
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sohan Jain
> Assignee: Sohan Jain
> Priority: Minor
> Attachments: HIVE-2188.1.patch
>
>
> This function would get multiple tables from the hive metastore as opposed to just one at a time, saving round trip time to the metastore.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2188) Add multi_get_table function in Hive
Metastore
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13045228#comment-13045228 ]
jiraposter@reviews.apache.org commented on HIVE-2188:
-----------------------------------------------------
bq. On 2011-06-06 22:18:08, Carl Steinbach wrote:
bq. > trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, line 1153
bq. > <https://reviews.apache.org/r/831/diff/2/?file=20589#file20589line1153>
bq. >
bq. > Maybe change this to "get_table_objects_by_name" in order to disambiguate from cases where we're returning only tables names, or applying a filter condition, etc. etc.
Sounds good.
bq. On 2011-06-06 22:18:08, Carl Steinbach wrote:
bq. > trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, line 1164
bq. > <https://reviews.apache.org/r/831/diff/2/?file=20589#file20589line1164>
bq. >
bq. > This should probably be InvalidOperationException instead of NoSuchObjectException.
bq. >
bq. > It might also be good to validate the dbname input parameter at this step, e.g. make sure it's not null and not an empty string.
Ah ok thanks; it felt strange to throw a NoSuchObjectException there.
bq. On 2011-06-06 22:18:08, Carl Steinbach wrote:
bq. > trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, line 1171
bq. > <https://reviews.apache.org/r/831/diff/2/?file=20589#file20589line1171>
bq. >
bq. > Failing the entire operation if a single table in the input list is not defined seems like a bad idea since we're throwing away work that will have to be repeated on the next call. Furthermore, the exception doesn't contain any information about which table(s) are not defined, so the client will have to fetch a table list again and use this to construct the list of input tables for the next get_multi_table() call. In the meantime it's possible that someone will drop a table in the list, which will invalidate the next call.
bq. >
bq. > I think it would be better to modify the contract to state that if a table on the input list is not found in the metastore, then the table definition will not be included in the result. This means that the function will return an empty list if none of the tables in the input list are found in the metastore.
bq. >
I see your concern, I'll change it to retrieve only tables that are in the metastore.
bq. On 2011-06-06 22:18:08, Carl Steinbach wrote:
bq. > trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java, line 818
bq. > <https://reviews.apache.org/r/831/diff/2/?file=20590#file20590line818>
bq. >
bq. > Only some callers will care about having this ordering property satisfied, so instead of penalizing every caller with this performance hit, maybe it would be better to let the caller take care of this?
bq. >
Sounds good. Removing the ordering especially makes sense if we cannot guarantee that we return *every* table the caller requested.
bq. On 2011-06-06 22:18:08, Carl Steinbach wrote:
bq. > trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java, line 802
bq. > <https://reviews.apache.org/r/831/diff/2/?file=20590#file20590line802>
bq. >
bq. > It would be good to first check if the DB exists, and throw UnknownDBException if it's not found.
Will do. As a side note, this check requires another db call, which may slow down the function a little.
- Sohan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/831/#review768
-----------------------------------------------------------
On 2011-06-07 01:11:44, Sohan Jain wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/831/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-06-07 01:11:44)
bq.
bq.
bq. Review request for hive, Paul Yang and Ashutosh Chauhan.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Created a function "multi_get_table" that retrieves multiple tables on one trip to the hive metastore, saving round trip time.
bq.
bq.
bq. This addresses bug HIVE-2188.
bq. https://issues.apache.org/jira/browse/HIVE-2188
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. trunk/metastore/if/hive_metastore.thrift 1130342
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1130342
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1130342
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1130342
bq. trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java 1130342
bq.
bq. Diff: https://reviews.apache.org/r/831/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Added a test case to testMetasore() in TestHiveServer. Also tested for speed improvements in a client session.
bq.
bq.
bq. Thanks,
bq.
bq. Sohan
bq.
bq.
> Add multi_get_table function in Hive Metastore
> ----------------------------------------------
>
> Key: HIVE-2188
> URL: https://issues.apache.org/jira/browse/HIVE-2188
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sohan Jain
> Assignee: Sohan Jain
> Priority: Minor
> Attachments: HIVE-2188.1.patch, HIVE-2188.3.patch
>
>
> This function would get multiple tables from the hive metastore as opposed to just one at a time, saving round trip time to the metastore.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2188) Add multi_get_table function in Hive
Metastore
Posted by "Sohan Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sohan Jain updated HIVE-2188:
-----------------------------
Status: Patch Available (was: Open)
> Add multi_get_table function in Hive Metastore
> ----------------------------------------------
>
> Key: HIVE-2188
> URL: https://issues.apache.org/jira/browse/HIVE-2188
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sohan Jain
> Assignee: Sohan Jain
> Priority: Minor
> Attachments: HIVE-2188.1.patch
>
>
> This function would get multiple tables from the hive metastore as opposed to just one at a time, saving round trip time to the metastore.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2188) Add multi_get_table function in Hive
Metastore
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13042527#comment-13042527 ]
jiraposter@reviews.apache.org commented on HIVE-2188:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/831/
-----------------------------------------------------------
Review request for hive and Paul Yang.
Summary
-------
Created a function "multi_get_table" that retrieves multiple tables on one trip to the hive metastore, saving round trip time.
This addresses bug HIVE-2188.
https://issues.apache.org/jira/browse/HIVE-2188
Diffs
-----
trunk/metastore/if/hive_metastore.thrift 1130342
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1130342
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1130342
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1130342
trunk/service/src/test/org/apache/hadoop/hive/service/TestHiveServer.java 1130342
Diff: https://reviews.apache.org/r/831/diff
Testing
-------
Added a test case to testMetasore() in TestHiveServer. Also tested for speed improvements in a client session.
Thanks,
Sohan
> Add multi_get_table function in Hive Metastore
> ----------------------------------------------
>
> Key: HIVE-2188
> URL: https://issues.apache.org/jira/browse/HIVE-2188
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sohan Jain
> Assignee: Sohan Jain
> Priority: Minor
> Attachments: HIVE-2188.1.patch
>
>
> This function would get multiple tables from the hive metastore as opposed to just one at a time, saving round trip time to the metastore.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2188) Add multi_get_table function in Hive
Metastore
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13046733#comment-13046733 ]
Carl Steinbach commented on HIVE-2188:
--------------------------------------
+1. Will commit if tests pass.
> Add multi_get_table function in Hive Metastore
> ----------------------------------------------
>
> Key: HIVE-2188
> URL: https://issues.apache.org/jira/browse/HIVE-2188
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sohan Jain
> Assignee: Sohan Jain
> Priority: Minor
> Attachments: HIVE-2188.1.patch, HIVE-2188.3.patch
>
>
> This function would get multiple tables from the hive metastore as opposed to just one at a time, saving round trip time to the metastore.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2188) Add multi_get_table function in Hive
Metastore
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13043117#comment-13043117 ]
jiraposter@reviews.apache.org commented on HIVE-2188:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/831/
-----------------------------------------------------------
(Updated 2011-06-02 23:01:00.830757)
Review request for hive, Paul Yang and Ashutosh Chauhan.
Summary
-------
Created a function "multi_get_table" that retrieves multiple tables on one trip to the hive metastore, saving round trip time.
This addresses bug HIVE-2188.
https://issues.apache.org/jira/browse/HIVE-2188
Diffs
-----
trunk/metastore/if/hive_metastore.thrift 1130342
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1130342
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1130342
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1130342
trunk/service/src/test/org/apache/hadoop/hive/service/TestHiveServer.java 1130342
Diff: https://reviews.apache.org/r/831/diff
Testing
-------
Added a test case to testMetasore() in TestHiveServer. Also tested for speed improvements in a client session.
Thanks,
Sohan
> Add multi_get_table function in Hive Metastore
> ----------------------------------------------
>
> Key: HIVE-2188
> URL: https://issues.apache.org/jira/browse/HIVE-2188
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sohan Jain
> Assignee: Sohan Jain
> Priority: Minor
> Attachments: HIVE-2188.1.patch
>
>
> This function would get multiple tables from the hive metastore as opposed to just one at a time, saving round trip time to the metastore.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2188) Add multi_get_table function in Hive
Metastore
Posted by "Sohan Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sohan Jain updated HIVE-2188:
-----------------------------
Attachment: HIVE-2188.1.patch
- Added multiGetTable function to the interface RawStore
- Implemented it in ObjectStore imitating the SQL "IN" operator in JDO
- added multi_get_table function to HiveMetaStore that is a wrapper to the RawStore function.
> Add multi_get_table function in Hive Metastore
> ----------------------------------------------
>
> Key: HIVE-2188
> URL: https://issues.apache.org/jira/browse/HIVE-2188
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sohan Jain
> Assignee: Sohan Jain
> Priority: Minor
> Attachments: HIVE-2188.1.patch
>
>
> This function would get multiple tables from the hive metastore as opposed to just one at a time, saving round trip time to the metastore.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2188) Add multi_get_table function in Hive
Metastore
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13045758#comment-13045758 ]
jiraposter@reviews.apache.org commented on HIVE-2188:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/831/
-----------------------------------------------------------
(Updated 2011-06-08 02:53:35.735457)
Review request for hive, Paul Yang and Ashutosh Chauhan.
Changes
-------
- added getTableObjectsByName to IMetaStoreClient and HiveMetaStoreClient
- modified the unit tests to reflect this change
Summary
-------
Created a function "multi_get_table" that retrieves multiple tables on one trip to the hive metastore, saving round trip time.
This addresses bug HIVE-2188.
https://issues.apache.org/jira/browse/HIVE-2188
Diffs (updated)
-----
trunk/metastore/if/hive_metastore.thrift 1133230
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1133230
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 1133230
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 1133230
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 1133230
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 1133230
trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java 1133230
Diff: https://reviews.apache.org/r/831/diff
Testing
-------
Added a test case to testMetasore() in TestHiveServer. Also tested for speed improvements in a client session.
Thanks,
Sohan
> Add multi_get_table function in Hive Metastore
> ----------------------------------------------
>
> Key: HIVE-2188
> URL: https://issues.apache.org/jira/browse/HIVE-2188
> Project: Hive
> Issue Type: New Feature
> Components: Metastore
> Reporter: Sohan Jain
> Assignee: Sohan Jain
> Priority: Minor
> Attachments: HIVE-2188.1.patch, HIVE-2188.3.patch
>
>
> This function would get multiple tables from the hive metastore as opposed to just one at a time, saving round trip time to the metastore.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira