You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Ashutosh Chauhan (JIRA)" <ji...@apache.org> on 2011/04/27 08:54:03 UTC
[jira] [Created] (HIVE-2135) Refactor HiveMetaStore.java
Refactor HiveMetaStore.java
---------------------------
Key: HIVE-2135
URL: https://issues.apache.org/jira/browse/HIVE-2135
Project: Hive
Issue Type: Improvement
Components: Metastore
Reporter: Ashutosh Chauhan
Assignee: Ashutosh Chauhan
Fix For: 0.8.0
HiveMetaStore.java is more then 3000 LOC which contains wide variety of code. It will be good to refactor it so that its easier to maintain.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2135) Refactor HiveMetaStore.java
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated HIVE-2135:
-----------------------------------
Status: Open (was: Patch Available)
> Refactor HiveMetaStore.java
> ---------------------------
>
> Key: HIVE-2135
> URL: https://issues.apache.org/jira/browse/HIVE-2135
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: refactor.patch
>
>
> HiveMetaStore.java is more then 3000 LOC which contains wide variety of code. It will be good to refactor it so that its easier to maintain.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2135) Refactor HiveMetaStore.java
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027144#comment-13027144 ]
Carl Steinbach commented on HIVE-2135:
--------------------------------------
Changes look good to me. Nice to see some cleanup work!
@John and Paul: Can one of you take a look at this and make sure it doesn't break anything on the FB side?
> Refactor HiveMetaStore.java
> ---------------------------
>
> Key: HIVE-2135
> URL: https://issues.apache.org/jira/browse/HIVE-2135
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: refactor.patch
>
>
> HiveMetaStore.java is more then 3000 LOC which contains wide variety of code. It will be good to refactor it so that its easier to maintain.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2135) Refactor HiveMetaStore.java
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13025936#comment-13025936 ]
jiraposter@reviews.apache.org commented on HIVE-2135:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/669/
-----------------------------------------------------------
(Updated 2011-04-27 18:24:42.458082)
Review request for hive, Carl Steinbach, John Sichi, and Paul Yang.
Changes
-------
Mistyped jira number.
Summary
-------
See HIVE-2135
This addresses bug HIVE-2135.
https://issues.apache.org/jira/browse/HIVE-2135
Diffs
-----
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1096976
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreCommand.java PRE-CREATION
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1096976
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/URLConnectionUpdater.java PRE-CREATION
Diff: https://reviews.apache.org/r/669/diff
Testing
-------
Since this is a refactoring patch, no new tests are required. Ran all the tests in metastore. All of them passed.
Thanks,
Ashutosh
> Refactor HiveMetaStore.java
> ---------------------------
>
> Key: HIVE-2135
> URL: https://issues.apache.org/jira/browse/HIVE-2135
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: refactor.patch
>
>
> HiveMetaStore.java is more then 3000 LOC which contains wide variety of code. It will be good to refactor it so that its easier to maintain.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2135) Refactor HiveMetaStore.java
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated HIVE-2135:
-----------------------------------
Attachment: refactor.patch
Major part of this refactoring is done using Eclipse.
> Refactor HiveMetaStore.java
> ---------------------------
>
> Key: HIVE-2135
> URL: https://issues.apache.org/jira/browse/HIVE-2135
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: refactor.patch
>
>
> HiveMetaStore.java is more then 3000 LOC which contains wide variety of code. It will be good to refactor it so that its easier to maintain.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-2135) Refactor HiveMetaStore.java
Posted by "Ashutosh Chauhan (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan resolved HIVE-2135.
------------------------------------
Resolution: Duplicate
This got taken care of in a better fashion in HIVE-2716.
> Refactor HiveMetaStore.java
> ---------------------------
>
> Key: HIVE-2135
> URL: https://issues.apache.org/jira/browse/HIVE-2135
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive_2135.patch, refactor.patch
>
>
> HiveMetaStore.java is more then 3000 LOC which contains wide variety of code. It will be good to refactor it so that its easier to maintain.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2135) Refactor HiveMetaStore.java
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027177#comment-13027177 ]
jiraposter@reviews.apache.org commented on HIVE-2135:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/669/#review622
-----------------------------------------------------------
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreCommand.java
<https://reviews.apache.org/r/669/#comment1265>
The example in this Javadoc needs to be updated to match the new invocation pattern. (Also, the original Javadoc was missing <pre> and <code> tags; can you add those in here so that the HTML will get generated correctly?)
But first...is the new invocation pattern actually better? It seems like the invocation has gotten a little more verbose since each call to executeWithRetry now has to pass in two arguments. Wouldn't it be better to leave the executeWithRetry method in the HiveMetaStore class so that it can do any common stuff like setting these as attributes on the command object?
Likewise, I'm not clear on how moving the thread-local access to the command object is an improvement. Seems to me it should stay in HiveMetaStore; again, you can pass the right object in from executeWithRetry.
- John
On 2011-04-27 18:24:42, Ashutosh Chauhan wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/669/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-04-27 18:24:42)
bq.
bq.
bq. Review request for hive, Carl Steinbach, John Sichi, and Paul Yang.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. See HIVE-2135
bq.
bq.
bq. This addresses bug HIVE-2135.
bq. https://issues.apache.org/jira/browse/HIVE-2135
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 1096976
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreCommand.java PRE-CREATION
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1096976
bq. trunk/metastore/src/java/org/apache/hadoop/hive/metastore/URLConnectionUpdater.java PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/669/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Since this is a refactoring patch, no new tests are required. Ran all the tests in metastore. All of them passed.
bq.
bq.
bq. Thanks,
bq.
bq. Ashutosh
bq.
bq.
> Refactor HiveMetaStore.java
> ---------------------------
>
> Key: HIVE-2135
> URL: https://issues.apache.org/jira/browse/HIVE-2135
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: refactor.patch
>
>
> HiveMetaStore.java is more then 3000 LOC which contains wide variety of code. It will be good to refactor it so that its easier to maintain.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2135) Refactor HiveMetaStore.java
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated HIVE-2135:
-----------------------------------
Attachment: hive_2135.patch
Objectstore, Metastore, Command, execution, retrials are distinct concepts which I think are too entwined in current implementation. It would be good to disentangle them from each other to make code more modular. I think its better to have a separate Command interface which different types of Command will implement. execute() should really be a part of Rawstore interface since different Objectstore can have different strategies for executing a command and may have different ways of doing retrials. Metastore should prepare a Command and call execute() on Rawstore to execute it. I am attaching an illustrative patch which sketches these ideas for create database command. Let me know if it makes sense.
> Refactor HiveMetaStore.java
> ---------------------------
>
> Key: HIVE-2135
> URL: https://issues.apache.org/jira/browse/HIVE-2135
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: hive_2135.patch, refactor.patch
>
>
> HiveMetaStore.java is more then 3000 LOC which contains wide variety of code. It will be good to refactor it so that its easier to maintain.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2135) Refactor HiveMetaStore.java
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated HIVE-2135:
-----------------------------------
Status: Patch Available (was: Open)
> Refactor HiveMetaStore.java
> ---------------------------
>
> Key: HIVE-2135
> URL: https://issues.apache.org/jira/browse/HIVE-2135
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Fix For: 0.8.0
>
> Attachments: refactor.patch
>
>
> HiveMetaStore.java is more then 3000 LOC which contains wide variety of code. It will be good to refactor it so that its easier to maintain.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2135) Refactor HiveMetaStore.java
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-2135:
---------------------------------
Fix Version/s: (was: 0.8.0)
> Refactor HiveMetaStore.java
> ---------------------------
>
> Key: HIVE-2135
> URL: https://issues.apache.org/jira/browse/HIVE-2135
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Ashutosh Chauhan
> Assignee: Ashutosh Chauhan
> Attachments: hive_2135.patch, refactor.patch
>
>
> HiveMetaStore.java is more then 3000 LOC which contains wide variety of code. It will be good to refactor it so that its easier to maintain.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira