You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Robert Joseph Evans (Created) (JIRA)" <ji...@apache.org> on 2011/12/09 18:21:40 UTC
[jira] [Created] (HADOOP-7901) Have Configuration use Read/Write
Locks
Have Configuration use Read/Write Locks
---------------------------------------
Key: HADOOP-7901
URL: https://issues.apache.org/jira/browse/HADOOP-7901
Project: Hadoop Common
Issue Type: Improvement
Components: conf
Affects Versions: 0.23.1
Reporter: Robert Joseph Evans
Assignee: Robert Joseph Evans
We can potentially improve performance by moving to read/write locks for configuration instead of the current synchronization.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7901) Have Configuration use Read/Write
Locks
Posted by "Robert Joseph Evans (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13166644#comment-13166644 ]
Robert Joseph Evans commented on HADOOP-7901:
---------------------------------------------
I have not measured any contention in the Configuration object. That is a very good point I really should not be doing optimizations without measuring first to know if they are needless or not. I really should rename this measure lock contention in Configuration object and then possibly use Read/Write Locks.
This was filed partly in response to MAPREDUCE-3519. There was a deadlock introduced because one thread grabbed a lock on a Configuration object and then tried to grab a different lock while a separate thread tried to grab the locks in reverse order. This seems especially odd to me because my gut feeling is that most of the time all we are trying to do is to read data from Configuration. Very rarely do we want to change it, so I thought that it would be a potential performance improvement, in addition to removing some of the potential for these deadlocks in the future.
So before doing any code changes I will measure the level of lock contention.
> Have Configuration use Read/Write Locks
> ---------------------------------------
>
> Key: HADOOP-7901
> URL: https://issues.apache.org/jira/browse/HADOOP-7901
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Affects Versions: 0.23.1
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
>
> We can potentially improve performance by moving to read/write locks for configuration instead of the current synchronization.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7901) Have Configuration use Read/Write
Locks
Posted by "Robert Joseph Evans (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated HADOOP-7901:
----------------------------------------
Target Version/s: 2.0.0, 3.0.0 (was: 0.23.2)
> Have Configuration use Read/Write Locks
> ---------------------------------------
>
> Key: HADOOP-7901
> URL: https://issues.apache.org/jira/browse/HADOOP-7901
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Affects Versions: 0.23.1
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
>
> We can potentially improve performance by moving to read/write locks for configuration instead of the current synchronization.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7901) Have Configuration use Read/Write
Locks
Posted by "Todd Lipcon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13166363#comment-13166363 ]
Todd Lipcon commented on HADOOP-7901:
-------------------------------------
Where are you seeing contention on the Configuration object? The downside of rwlock is (a) increased memory usage, and (b) higher constant overhead, in my experience. Not that I'm against it, just would like to know more about the motivation.
> Have Configuration use Read/Write Locks
> ---------------------------------------
>
> Key: HADOOP-7901
> URL: https://issues.apache.org/jira/browse/HADOOP-7901
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Affects Versions: 0.23.1
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
>
> We can potentially improve performance by moving to read/write locks for configuration instead of the current synchronization.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira