You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Navis (JIRA)" <ji...@apache.org> on 2012/10/04 08:45:07 UTC
[jira] [Created] (HIVE-3531) Simple lock manager for dedicated hive
server
Navis created HIVE-3531:
---------------------------
Summary: Simple lock manager for dedicated hive server
Key: HIVE-3531
URL: https://issues.apache.org/jira/browse/HIVE-3531
Project: Hive
Issue Type: Improvement
Components: Server Infrastructure
Reporter: Navis
Assignee: Navis
Priority: Trivial
In many cases, we uses hive server as a sole proxy for executing all the queries. For that, current default lock manager based on zookeeper seemed a little heavy. Simple in-memory lock manager could be enough.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3531) Simple lock manager for dedicated
hive server
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13505431#comment-13505431 ]
Hudson commented on HIVE-3531:
------------------------------
Integrated in Hive-trunk-h0.21 #1822 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1822/])
HIVE-3531 [jira] Simple lock manager for dedicated hive server
(Navis Ryu via Carl Steinbach)
Summary:
DPAL-1906 Implement simple lock manager for hive server
In many cases, we uses hive server as a sole proxy for executing all the queries. For that, current default lock manager based on zookeeper seemed a little heavy. Simple in-memory lock manager could be enough.
Test Plan: TestDedicatedLockManager
Reviewers: JIRA, cwsteinbach
Reviewed By: cwsteinbach
Differential Revision: https://reviews.facebook.net/D5871 (Revision 1414590)
Result = ABORTED
cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1414590
Files :
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/lockmgr/EmbeddedLockManager.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/lockmgr/HiveLockObject.java
* /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/lockmgr
* /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/lockmgr/TestEmbeddedLockManager.java
> Simple lock manager for dedicated hive server
> ---------------------------------------------
>
> Key: HIVE-3531
> URL: https://issues.apache.org/jira/browse/HIVE-3531
> Project: Hive
> Issue Type: Improvement
> Components: Locking, Server Infrastructure
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Fix For: 0.11
>
> Attachments: HIVE-3531.D5871.1.patch, HIVE-3531.D5871.2.patch, HIVE-3531.D5871.3.patch
>
>
> In many cases, we uses hive server as a sole proxy for executing all the queries. For that, current default lock manager based on zookeeper seemed a little heavy. Simple in-memory lock manager could be enough.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-3531) Simple lock manager for dedicated
hive server
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach resolved HIVE-3531.
----------------------------------
Resolution: Fixed
Fix Version/s: 0.11
Hadoop Flags: Reviewed
Committed to trunk. Thanks Navis!
> Simple lock manager for dedicated hive server
> ---------------------------------------------
>
> Key: HIVE-3531
> URL: https://issues.apache.org/jira/browse/HIVE-3531
> Project: Hive
> Issue Type: Improvement
> Components: Locking, Server Infrastructure
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Fix For: 0.11
>
> Attachments: HIVE-3531.D5871.1.patch, HIVE-3531.D5871.2.patch, HIVE-3531.D5871.3.patch
>
>
> In many cases, we uses hive server as a sole proxy for executing all the queries. For that, current default lock manager based on zookeeper seemed a little heavy. Simple in-memory lock manager could be enough.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3531) Simple lock manager for dedicated
hive server
Posted by "Phabricator (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504044#comment-13504044 ]
Phabricator commented on HIVE-3531:
-----------------------------------
cwsteinbach has accepted the revision "HIVE-3531 [jira] Simple lock manager for dedicated hive server".
REVISION DETAIL
https://reviews.facebook.net/D5871
BRANCH
DPAL-1906
To: JIRA, cwsteinbach, navis
> Simple lock manager for dedicated hive server
> ---------------------------------------------
>
> Key: HIVE-3531
> URL: https://issues.apache.org/jira/browse/HIVE-3531
> Project: Hive
> Issue Type: Improvement
> Components: Locking, Server Infrastructure
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3531.D5871.1.patch, HIVE-3531.D5871.2.patch, HIVE-3531.D5871.3.patch
>
>
> In many cases, we uses hive server as a sole proxy for executing all the queries. For that, current default lock manager based on zookeeper seemed a little heavy. Simple in-memory lock manager could be enough.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3531) Simple lock manager for dedicated
hive server
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504040#comment-13504040 ]
Carl Steinbach commented on HIVE-3531:
--------------------------------------
+1. Will commit if tests pass.
> Simple lock manager for dedicated hive server
> ---------------------------------------------
>
> Key: HIVE-3531
> URL: https://issues.apache.org/jira/browse/HIVE-3531
> Project: Hive
> Issue Type: Improvement
> Components: Locking, Server Infrastructure
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3531.D5871.1.patch, HIVE-3531.D5871.2.patch, HIVE-3531.D5871.3.patch
>
>
> In many cases, we uses hive server as a sole proxy for executing all the queries. For that, current default lock manager based on zookeeper seemed a little heavy. Simple in-memory lock manager could be enough.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3531) Simple lock manager for dedicated
hive server
Posted by "Phabricator (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503583#comment-13503583 ]
Phabricator commented on HIVE-3531:
-----------------------------------
navis has commented on the revision "HIVE-3531 [jira] Simple lock manager for dedicated hive server".
Addressed comments. And i think default values for this needed to be tuned.
REVISION DETAIL
https://reviews.facebook.net/D5871
To: JIRA, cwsteinbach, navis
> Simple lock manager for dedicated hive server
> ---------------------------------------------
>
> Key: HIVE-3531
> URL: https://issues.apache.org/jira/browse/HIVE-3531
> Project: Hive
> Issue Type: Improvement
> Components: Locking, Server Infrastructure
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3531.D5871.1.patch, HIVE-3531.D5871.2.patch
>
>
> In many cases, we uses hive server as a sole proxy for executing all the queries. For that, current default lock manager based on zookeeper seemed a little heavy. Simple in-memory lock manager could be enough.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3531) Simple lock manager for dedicated hive
server
Posted by "Phabricator (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-3531:
------------------------------
Attachment: HIVE-3531.D5871.2.patch
navis updated the revision "HIVE-3531 [jira] Simple lock manager for dedicated hive server".
Reviewers: JIRA, cwsteinbach
Addressed comments & rebased to trunk
REVISION DETAIL
https://reviews.facebook.net/D5871
AFFECTED FILES
ql/src/java/org/apache/hadoop/hive/ql/lockmgr/EmbeddedLockManager.java
ql/src/java/org/apache/hadoop/hive/ql/lockmgr/HiveLockObject.java
ql/src/test/org/apache/hadoop/hive/ql/lockmgr/TestEmbeddedLockManager.java
To: JIRA, cwsteinbach, navis
> Simple lock manager for dedicated hive server
> ---------------------------------------------
>
> Key: HIVE-3531
> URL: https://issues.apache.org/jira/browse/HIVE-3531
> Project: Hive
> Issue Type: Improvement
> Components: Locking, Server Infrastructure
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3531.D5871.1.patch, HIVE-3531.D5871.2.patch
>
>
> In many cases, we uses hive server as a sole proxy for executing all the queries. For that, current default lock manager based on zookeeper seemed a little heavy. Simple in-memory lock manager could be enough.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3531) Simple lock manager for dedicated
hive server
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503655#comment-13503655 ]
Carl Steinbach commented on HIVE-3531:
--------------------------------------
@Navis: Can you please add an ASF license header to TestEmbeddedLockManager? Looks good otherwise.
> Simple lock manager for dedicated hive server
> ---------------------------------------------
>
> Key: HIVE-3531
> URL: https://issues.apache.org/jira/browse/HIVE-3531
> Project: Hive
> Issue Type: Improvement
> Components: Locking, Server Infrastructure
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3531.D5871.1.patch, HIVE-3531.D5871.2.patch
>
>
> In many cases, we uses hive server as a sole proxy for executing all the queries. For that, current default lock manager based on zookeeper seemed a little heavy. Simple in-memory lock manager could be enough.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3531) Simple lock manager for dedicated hive
server
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-3531:
---------------------------------
Component/s: Locking
> Simple lock manager for dedicated hive server
> ---------------------------------------------
>
> Key: HIVE-3531
> URL: https://issues.apache.org/jira/browse/HIVE-3531
> Project: Hive
> Issue Type: Improvement
> Components: Locking, Server Infrastructure
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3531.D5871.1.patch
>
>
> In many cases, we uses hive server as a sole proxy for executing all the queries. For that, current default lock manager based on zookeeper seemed a little heavy. Simple in-memory lock manager could be enough.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3531) Simple lock manager for dedicated
hive server
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13497827#comment-13497827 ]
Carl Steinbach commented on HIVE-3531:
--------------------------------------
@Navis: I added some comments on phabricator. Please take a look when you have time. Thanks.
> Simple lock manager for dedicated hive server
> ---------------------------------------------
>
> Key: HIVE-3531
> URL: https://issues.apache.org/jira/browse/HIVE-3531
> Project: Hive
> Issue Type: Improvement
> Components: Locking, Server Infrastructure
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3531.D5871.1.patch
>
>
> In many cases, we uses hive server as a sole proxy for executing all the queries. For that, current default lock manager based on zookeeper seemed a little heavy. Simple in-memory lock manager could be enough.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3531) Simple lock manager for dedicated
hive server
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469552#comment-13469552 ]
Carl Steinbach commented on HIVE-3531:
--------------------------------------
@Navis: This is awesome. We were planning to write something like this for HiveServer2, but you beat us to it :)
I'll try to review this in the next day or so. Thanks!
> Simple lock manager for dedicated hive server
> ---------------------------------------------
>
> Key: HIVE-3531
> URL: https://issues.apache.org/jira/browse/HIVE-3531
> Project: Hive
> Issue Type: Improvement
> Components: Server Infrastructure
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3531.D5871.1.patch
>
>
> In many cases, we uses hive server as a sole proxy for executing all the queries. For that, current default lock manager based on zookeeper seemed a little heavy. Simple in-memory lock manager could be enough.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3531) Simple lock manager for dedicated hive
server
Posted by "Phabricator (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-3531:
------------------------------
Attachment: HIVE-3531.D5871.1.patch
navis requested code review of "HIVE-3531 [jira] Simple lock manager for dedicated hive server".
Reviewers: JIRA
DPAL-1906 Implement simple lock manager for hive server
In many cases, we uses hive server as a sole proxy for executing all the queries. For that, current default lock manager based on zookeeper seemed a little heavy. Simple in-memory lock manager could be enough.
TEST PLAN
TestDedicatedLockManager
REVISION DETAIL
https://reviews.facebook.net/D5871
AFFECTED FILES
ql/src/java/org/apache/hadoop/hive/ql/lockmgr/DedicatedLockManager.java
ql/src/java/org/apache/hadoop/hive/ql/lockmgr/HiveLockObject.java
ql/src/java/org/apache/hadoop/hive/ql/lockmgr/InMemoryLockManager.java
ql/src/test/org/apache/hadoop/hive/ql/lockmgr/TestDedicatedLockManager.java
MANAGE HERALD DIFFERENTIAL RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/13923/
To: JIRA, navis
> Simple lock manager for dedicated hive server
> ---------------------------------------------
>
> Key: HIVE-3531
> URL: https://issues.apache.org/jira/browse/HIVE-3531
> Project: Hive
> Issue Type: Improvement
> Components: Server Infrastructure
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3531.D5871.1.patch
>
>
> In many cases, we uses hive server as a sole proxy for executing all the queries. For that, current default lock manager based on zookeeper seemed a little heavy. Simple in-memory lock manager could be enough.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3531) Simple lock manager for dedicated
hive server
Posted by "Phabricator (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503654#comment-13503654 ]
Phabricator commented on HIVE-3531:
-----------------------------------
cwsteinbach has requested changes to the revision "HIVE-3531 [jira] Simple lock manager for dedicated hive server".
INLINE COMMENTS
ql/src/test/org/apache/hadoop/hive/ql/lockmgr/TestEmbeddedLockManager.java:1 Please add an ASF license header.
REVISION DETAIL
https://reviews.facebook.net/D5871
BRANCH
DPAL-1906
To: JIRA, cwsteinbach, navis
> Simple lock manager for dedicated hive server
> ---------------------------------------------
>
> Key: HIVE-3531
> URL: https://issues.apache.org/jira/browse/HIVE-3531
> Project: Hive
> Issue Type: Improvement
> Components: Locking, Server Infrastructure
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3531.D5871.1.patch, HIVE-3531.D5871.2.patch
>
>
> In many cases, we uses hive server as a sole proxy for executing all the queries. For that, current default lock manager based on zookeeper seemed a little heavy. Simple in-memory lock manager could be enough.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3531) Simple lock manager for dedicated hive
server
Posted by "Phabricator (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-3531:
------------------------------
Attachment: HIVE-3531.D5871.3.patch
navis updated the revision "HIVE-3531 [jira] Simple lock manager for dedicated hive server".
Reviewers: JIRA, cwsteinbach
added ASF header. sorry again
REVISION DETAIL
https://reviews.facebook.net/D5871
AFFECTED FILES
ql/src/java/org/apache/hadoop/hive/ql/lockmgr/EmbeddedLockManager.java
ql/src/java/org/apache/hadoop/hive/ql/lockmgr/HiveLockObject.java
ql/src/test/org/apache/hadoop/hive/ql/lockmgr/TestEmbeddedLockManager.java
To: JIRA, cwsteinbach, navis
> Simple lock manager for dedicated hive server
> ---------------------------------------------
>
> Key: HIVE-3531
> URL: https://issues.apache.org/jira/browse/HIVE-3531
> Project: Hive
> Issue Type: Improvement
> Components: Locking, Server Infrastructure
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3531.D5871.1.patch, HIVE-3531.D5871.2.patch, HIVE-3531.D5871.3.patch
>
>
> In many cases, we uses hive server as a sole proxy for executing all the queries. For that, current default lock manager based on zookeeper seemed a little heavy. Simple in-memory lock manager could be enough.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3531) Simple lock manager for dedicated
hive server
Posted by "Navis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469857#comment-13469857 ]
Navis commented on HIVE-3531:
-----------------------------
@Edward: As stated in description, we delegates execution of all the queries to a single hive server. In this case, external lock manager like a zookeeper could be used but it's not required thing. But it does not mean locking itself is not needed. Multiple queries could be executed concurrently and still need a appropriate locking system.
I didn't think of locking mechanism of hive deeply yet. It's just in-memory version of zookeeper lock manager. The part "locking does not working well" in your comment should be an another issue.
> Simple lock manager for dedicated hive server
> ---------------------------------------------
>
> Key: HIVE-3531
> URL: https://issues.apache.org/jira/browse/HIVE-3531
> Project: Hive
> Issue Type: Improvement
> Components: Server Infrastructure
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3531.D5871.1.patch
>
>
> In many cases, we uses hive server as a sole proxy for executing all the queries. For that, current default lock manager based on zookeeper seemed a little heavy. Simple in-memory lock manager could be enough.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3531) Simple lock manager for dedicated
hive server
Posted by "Phabricator (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503739#comment-13503739 ]
Phabricator commented on HIVE-3531:
-----------------------------------
navis has commented on the revision "HIVE-3531 [jira] Simple lock manager for dedicated hive server".
INLINE COMMENTS
ql/src/test/org/apache/hadoop/hive/ql/lockmgr/TestEmbeddedLockManager.java:1 I remember I've added it, but missed again. Sorry.
REVISION DETAIL
https://reviews.facebook.net/D5871
BRANCH
DPAL-1906
To: JIRA, cwsteinbach, navis
> Simple lock manager for dedicated hive server
> ---------------------------------------------
>
> Key: HIVE-3531
> URL: https://issues.apache.org/jira/browse/HIVE-3531
> Project: Hive
> Issue Type: Improvement
> Components: Locking, Server Infrastructure
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3531.D5871.1.patch, HIVE-3531.D5871.2.patch
>
>
> In many cases, we uses hive server as a sole proxy for executing all the queries. For that, current default lock manager based on zookeeper seemed a little heavy. Simple in-memory lock manager could be enough.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3531) Simple lock manager for dedicated
hive server
Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469550#comment-13469550 ]
Edward Capriolo commented on HIVE-3531:
---------------------------------------
[~navis] Interesting. I do know that the locking does not work well with thrift service currently as I have had issues with it.
What is your use case for this? Why would you want a lock manager that does not actually work? One possibility is that it may be much nicer to make this the default for unit tests, but other then that I can not think of one.
> Simple lock manager for dedicated hive server
> ---------------------------------------------
>
> Key: HIVE-3531
> URL: https://issues.apache.org/jira/browse/HIVE-3531
> Project: Hive
> Issue Type: Improvement
> Components: Server Infrastructure
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3531.D5871.1.patch
>
>
> In many cases, we uses hive server as a sole proxy for executing all the queries. For that, current default lock manager based on zookeeper seemed a little heavy. Simple in-memory lock manager could be enough.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3531) Simple lock manager for dedicated
hive server
Posted by "Phabricator (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13497830#comment-13497830 ]
Phabricator commented on HIVE-3531:
-----------------------------------
cwsteinbach has requested changes to the revision "HIVE-3531 [jira] Simple lock manager for dedicated hive server".
INLINE COMMENTS
ql/src/test/org/apache/hadoop/hive/ql/lockmgr/TestDedicatedLockManager.java:1 Missing ASF license header.
ql/src/java/org/apache/hadoop/hive/ql/lockmgr/DedicatedLockManager.java:25 Please consider changing the name to EmbeddedLockManager.
ql/src/java/org/apache/hadoop/hive/ql/lockmgr/DedicatedLockManager.java:27 I'm not sure that creating a separate class for the InMemoryLockManager code makes sense. Please consider removing InMemoryLockManager and moving that code here.
ql/src/java/org/apache/hadoop/hive/ql/lockmgr/InMemoryLockManager.java:145 Please log this exception.
REVISION DETAIL
https://reviews.facebook.net/D5871
BRANCH
DPAL-1906
To: JIRA, cwsteinbach, navis
> Simple lock manager for dedicated hive server
> ---------------------------------------------
>
> Key: HIVE-3531
> URL: https://issues.apache.org/jira/browse/HIVE-3531
> Project: Hive
> Issue Type: Improvement
> Components: Locking, Server Infrastructure
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3531.D5871.1.patch
>
>
> In many cases, we uses hive server as a sole proxy for executing all the queries. For that, current default lock manager based on zookeeper seemed a little heavy. Simple in-memory lock manager could be enough.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3531) Simple lock manager for dedicated
hive server
Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469559#comment-13469559 ]
Edward Capriolo commented on HIVE-3531:
---------------------------------------
Why wouldnt I just want to turn that off?
> Simple lock manager for dedicated hive server
> ---------------------------------------------
>
> Key: HIVE-3531
> URL: https://issues.apache.org/jira/browse/HIVE-3531
> Project: Hive
> Issue Type: Improvement
> Components: Server Infrastructure
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3531.D5871.1.patch
>
>
> In many cases, we uses hive server as a sole proxy for executing all the queries. For that, current default lock manager based on zookeeper seemed a little heavy. Simple in-memory lock manager could be enough.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3531) Simple lock manager for dedicated
hive server
Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469558#comment-13469558 ]
Edward Capriolo commented on HIVE-3531:
---------------------------------------
@Carl Why? See my comments above. I am worried about code bloat in hive (No offense Navis your code is great). What is the case for having a lock manager that only does half the things a lock manager is supposed to do?
> Simple lock manager for dedicated hive server
> ---------------------------------------------
>
> Key: HIVE-3531
> URL: https://issues.apache.org/jira/browse/HIVE-3531
> Project: Hive
> Issue Type: Improvement
> Components: Server Infrastructure
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-3531.D5871.1.patch
>
>
> In many cases, we uses hive server as a sole proxy for executing all the queries. For that, current default lock manager based on zookeeper seemed a little heavy. Simple in-memory lock manager could be enough.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3531) Simple lock manager for dedicated
hive server
Posted by "Phabricator (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13505315#comment-13505315 ]
Phabricator commented on HIVE-3531:
-----------------------------------
navis has closed the revision "HIVE-3531 [jira] Simple lock manager for dedicated hive server".
Closed by cws.
CHANGED PRIOR TO COMMIT
https://reviews.facebook.net/D5871?vs=22401&id=22593#differential-review-toc
REVISION DETAIL
https://reviews.facebook.net/D5871
COMMIT
https://reviews.facebook.net/rHIVE1414590
To: JIRA, cwsteinbach, navis
> Simple lock manager for dedicated hive server
> ---------------------------------------------
>
> Key: HIVE-3531
> URL: https://issues.apache.org/jira/browse/HIVE-3531
> Project: Hive
> Issue Type: Improvement
> Components: Locking, Server Infrastructure
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Fix For: 0.11
>
> Attachments: HIVE-3531.D5871.1.patch, HIVE-3531.D5871.2.patch, HIVE-3531.D5871.3.patch
>
>
> In many cases, we uses hive server as a sole proxy for executing all the queries. For that, current default lock manager based on zookeeper seemed a little heavy. Simple in-memory lock manager could be enough.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira