You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Sameer Paranjpye (JIRA)" <ji...@apache.org> on 2006/11/16 00:42:39 UTC
[jira] Created: (HADOOP-726) HDFS locking mechanisms should be
simplified or removed
HDFS locking mechanisms should be simplified or removed
-------------------------------------------------------
Key: HADOOP-726
URL: http://issues.apache.org/jira/browse/HADOOP-726
Project: Hadoop
Issue Type: Improvement
Components: dfs
Reporter: Sameer Paranjpye
Priority: Minor
HDFS includes a locking mechanism that allows clients to lock files and directories in the filesystem. This requires that the Namenode, in addition to all other management of filesystem state also has to act like a lock manager on behalf of clients. Rather than burden the Namenode with this Hadoop should include a distinct general purpose distributed lock manager.
In the interim, the locking functionality on the filesystem should be simplified to permit only file locks or removed altogether. At this point, removing or simplifying the functionality is not likely to break client code, removing it later will be much harder. In any case, locks in HDFS don't work very well, see HADOOP-656. Also, directory locks are overkill most UNIX fses don't support it.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-726) HDFS locking mechanisms should be
simplified or removed
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated HADOOP-726:
--------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this. Thanks, Raghu!
> HDFS locking mechanisms should be simplified or removed
> -------------------------------------------------------
>
> Key: HADOOP-726
> URL: https://issues.apache.org/jira/browse/HADOOP-726
> Project: Hadoop
> Issue Type: Improvement
> Components: dfs
> Reporter: Sameer Paranjpye
> Assigned To: Raghu Angadi
> Priority: Minor
> Fix For: 0.11.0
>
> Attachments: HADOOP-726-1.patch, HADOOP-726-2.patch, remove-fs-locks.patch
>
>
> HDFS includes a locking mechanism that allows clients to lock files and directories in the filesystem. This requires that the Namenode, in addition to all other management of filesystem state also has to act like a lock manager on behalf of clients. Rather than burden the Namenode with this Hadoop should include a distinct general purpose distributed lock manager.
> In the interim, the locking functionality on the filesystem should be simplified to permit only file locks or removed altogether. At this point, removing or simplifying the functionality is not likely to break client code, removing it later will be much harder. In any case, locks in HDFS don't work very well, see HADOOP-656. Also, directory locks are overkill most UNIX fses don't support it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-726) HDFS locking mechanisms should be
simplified or removed
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated HADOOP-726:
--------------------------------
Status: Open (was: Patch Available)
I think what's most important is to deprecate the public methods. You've deprecated a number of package-private methods, and left some prominent public methods, like FileSystem#lock(), S3FileSystem#lock() and LocalFileSystem#lock(). These should also be deprecated. A separate issue should be created that removes these methods and their implementations in the next release.
> HDFS locking mechanisms should be simplified or removed
> -------------------------------------------------------
>
> Key: HADOOP-726
> URL: https://issues.apache.org/jira/browse/HADOOP-726
> Project: Hadoop
> Issue Type: Improvement
> Components: dfs
> Reporter: Sameer Paranjpye
> Assigned To: Raghu Angadi
> Priority: Minor
> Attachments: HADOOP-726-1.patch, remove-fs-locks.patch
>
>
> HDFS includes a locking mechanism that allows clients to lock files and directories in the filesystem. This requires that the Namenode, in addition to all other management of filesystem state also has to act like a lock manager on behalf of clients. Rather than burden the Namenode with this Hadoop should include a distinct general purpose distributed lock manager.
> In the interim, the locking functionality on the filesystem should be simplified to permit only file locks or removed altogether. At this point, removing or simplifying the functionality is not likely to break client code, removing it later will be much harder. In any case, locks in HDFS don't work very well, see HADOOP-656. Also, directory locks are overkill most UNIX fses don't support it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-726) HDFS locking mechanisms should be
simplified or removed
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-726:
--------------------------------
Fix Version/s: 0.11.0
Status: Patch Available (was: Open)
> HDFS locking mechanisms should be simplified or removed
> -------------------------------------------------------
>
> Key: HADOOP-726
> URL: https://issues.apache.org/jira/browse/HADOOP-726
> Project: Hadoop
> Issue Type: Improvement
> Components: dfs
> Reporter: Sameer Paranjpye
> Assigned To: Raghu Angadi
> Priority: Minor
> Fix For: 0.11.0
>
> Attachments: HADOOP-726-1.patch, HADOOP-726-2.patch, remove-fs-locks.patch
>
>
> HDFS includes a locking mechanism that allows clients to lock files and directories in the filesystem. This requires that the Namenode, in addition to all other management of filesystem state also has to act like a lock manager on behalf of clients. Rather than burden the Namenode with this Hadoop should include a distinct general purpose distributed lock manager.
> In the interim, the locking functionality on the filesystem should be simplified to permit only file locks or removed altogether. At this point, removing or simplifying the functionality is not likely to break client code, removing it later will be much harder. In any case, locks in HDFS don't work very well, see HADOOP-656. Also, directory locks are overkill most UNIX fses don't support it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-726) HDFS locking mechanisms should be
simplified or removed
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-726:
--------------------------------
Attachment: HADOOP-726-1.patch
HADOOP-726.patch marks locking functions @Deprecated.
> HDFS locking mechanisms should be simplified or removed
> -------------------------------------------------------
>
> Key: HADOOP-726
> URL: https://issues.apache.org/jira/browse/HADOOP-726
> Project: Hadoop
> Issue Type: Improvement
> Components: dfs
> Reporter: Sameer Paranjpye
> Assigned To: Raghu Angadi
> Priority: Minor
> Attachments: HADOOP-726-1.patch, remove-fs-locks.patch
>
>
> HDFS includes a locking mechanism that allows clients to lock files and directories in the filesystem. This requires that the Namenode, in addition to all other management of filesystem state also has to act like a lock manager on behalf of clients. Rather than burden the Namenode with this Hadoop should include a distinct general purpose distributed lock manager.
> In the interim, the locking functionality on the filesystem should be simplified to permit only file locks or removed altogether. At this point, removing or simplifying the functionality is not likely to break client code, removing it later will be much harder. In any case, locks in HDFS don't work very well, see HADOOP-656. Also, directory locks are overkill most UNIX fses don't support it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-726) HDFS locking mechanisms should be
simplified or removed
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi reassigned HADOOP-726:
-----------------------------------
Assignee: Raghu Angadi (was: Sameer Paranjpye)
> HDFS locking mechanisms should be simplified or removed
> -------------------------------------------------------
>
> Key: HADOOP-726
> URL: https://issues.apache.org/jira/browse/HADOOP-726
> Project: Hadoop
> Issue Type: Improvement
> Components: dfs
> Reporter: Sameer Paranjpye
> Assigned To: Raghu Angadi
> Priority: Minor
>
> HDFS includes a locking mechanism that allows clients to lock files and directories in the filesystem. This requires that the Namenode, in addition to all other management of filesystem state also has to act like a lock manager on behalf of clients. Rather than burden the Namenode with this Hadoop should include a distinct general purpose distributed lock manager.
> In the interim, the locking functionality on the filesystem should be simplified to permit only file locks or removed altogether. At this point, removing or simplifying the functionality is not likely to break client code, removing it later will be much harder. In any case, locks in HDFS don't work very well, see HADOOP-656. Also, directory locks are overkill most UNIX fses don't support it.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-726) HDFS locking mechanisms should be
simplified or removed
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12466186 ]
Raghu Angadi commented on HADOOP-726:
-------------------------------------
Attached remove-fs-locks.patch. It is not meant for inclusion. Initially we just want to deprecate the locking interface and remove the functionality later. Before deprecating I removed the locks to verify hadoop passes build tests.
I will submit another patch that deprecates the inteface.
> HDFS locking mechanisms should be simplified or removed
> -------------------------------------------------------
>
> Key: HADOOP-726
> URL: https://issues.apache.org/jira/browse/HADOOP-726
> Project: Hadoop
> Issue Type: Improvement
> Components: dfs
> Reporter: Sameer Paranjpye
> Assigned To: Raghu Angadi
> Priority: Minor
> Attachments: remove-fs-locks.patch
>
>
> HDFS includes a locking mechanism that allows clients to lock files and directories in the filesystem. This requires that the Namenode, in addition to all other management of filesystem state also has to act like a lock manager on behalf of clients. Rather than burden the Namenode with this Hadoop should include a distinct general purpose distributed lock manager.
> In the interim, the locking functionality on the filesystem should be simplified to permit only file locks or removed altogether. At this point, removing or simplifying the functionality is not likely to break client code, removing it later will be much harder. In any case, locks in HDFS don't work very well, see HADOOP-656. Also, directory locks are overkill most UNIX fses don't support it.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-726) HDFS locking mechanisms should be
simplified or removed
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12466828 ]
Hadoop QA commented on HADOOP-726:
----------------------------------
+1, because http://issues.apache.org/jira/secure/attachment/12349472/HADOOP-726-1.patch applied and successfully tested against trunk revision r498829.
> HDFS locking mechanisms should be simplified or removed
> -------------------------------------------------------
>
> Key: HADOOP-726
> URL: https://issues.apache.org/jira/browse/HADOOP-726
> Project: Hadoop
> Issue Type: Improvement
> Components: dfs
> Reporter: Sameer Paranjpye
> Assigned To: Raghu Angadi
> Priority: Minor
> Attachments: HADOOP-726-1.patch, remove-fs-locks.patch
>
>
> HDFS includes a locking mechanism that allows clients to lock files and directories in the filesystem. This requires that the Namenode, in addition to all other management of filesystem state also has to act like a lock manager on behalf of clients. Rather than burden the Namenode with this Hadoop should include a distinct general purpose distributed lock manager.
> In the interim, the locking functionality on the filesystem should be simplified to permit only file locks or removed altogether. At this point, removing or simplifying the functionality is not likely to break client code, removing it later will be much harder. In any case, locks in HDFS don't work very well, see HADOOP-656. Also, directory locks are overkill most UNIX fses don't support it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-726) HDFS locking mechanisms should be
simplified or removed
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-726:
--------------------------------
Attachment: remove-fs-locks.patch
> HDFS locking mechanisms should be simplified or removed
> -------------------------------------------------------
>
> Key: HADOOP-726
> URL: https://issues.apache.org/jira/browse/HADOOP-726
> Project: Hadoop
> Issue Type: Improvement
> Components: dfs
> Reporter: Sameer Paranjpye
> Assigned To: Raghu Angadi
> Priority: Minor
> Attachments: remove-fs-locks.patch
>
>
> HDFS includes a locking mechanism that allows clients to lock files and directories in the filesystem. This requires that the Namenode, in addition to all other management of filesystem state also has to act like a lock manager on behalf of clients. Rather than burden the Namenode with this Hadoop should include a distinct general purpose distributed lock manager.
> In the interim, the locking functionality on the filesystem should be simplified to permit only file locks or removed altogether. At this point, removing or simplifying the functionality is not likely to break client code, removing it later will be much harder. In any case, locks in HDFS don't work very well, see HADOOP-656. Also, directory locks are overkill most UNIX fses don't support it.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-726) HDFS locking mechanisms should be
simplified or removed
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-726:
--------------------------------
Attachment: HADOOP-726-2.patch
Thanks Doug. 2.patch depcrecates the public lock / release() interface for FileSystem and other related classes. verified that this does not introduce any deprecation warning in compiler.
I will file another issue for removing the functionality.
> HDFS locking mechanisms should be simplified or removed
> -------------------------------------------------------
>
> Key: HADOOP-726
> URL: https://issues.apache.org/jira/browse/HADOOP-726
> Project: Hadoop
> Issue Type: Improvement
> Components: dfs
> Reporter: Sameer Paranjpye
> Assigned To: Raghu Angadi
> Priority: Minor
> Attachments: HADOOP-726-1.patch, HADOOP-726-2.patch, remove-fs-locks.patch
>
>
> HDFS includes a locking mechanism that allows clients to lock files and directories in the filesystem. This requires that the Namenode, in addition to all other management of filesystem state also has to act like a lock manager on behalf of clients. Rather than burden the Namenode with this Hadoop should include a distinct general purpose distributed lock manager.
> In the interim, the locking functionality on the filesystem should be simplified to permit only file locks or removed altogether. At this point, removing or simplifying the functionality is not likely to break client code, removing it later will be much harder. In any case, locks in HDFS don't work very well, see HADOOP-656. Also, directory locks are overkill most UNIX fses don't support it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-726) HDFS locking mechanisms should be
simplified or removed
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-726:
--------------------------------
Status: Patch Available (was: Open)
> HDFS locking mechanisms should be simplified or removed
> -------------------------------------------------------
>
> Key: HADOOP-726
> URL: https://issues.apache.org/jira/browse/HADOOP-726
> Project: Hadoop
> Issue Type: Improvement
> Components: dfs
> Reporter: Sameer Paranjpye
> Assigned To: Raghu Angadi
> Priority: Minor
> Attachments: HADOOP-726-1.patch, remove-fs-locks.patch
>
>
> HDFS includes a locking mechanism that allows clients to lock files and directories in the filesystem. This requires that the Namenode, in addition to all other management of filesystem state also has to act like a lock manager on behalf of clients. Rather than burden the Namenode with this Hadoop should include a distinct general purpose distributed lock manager.
> In the interim, the locking functionality on the filesystem should be simplified to permit only file locks or removed altogether. At this point, removing or simplifying the functionality is not likely to break client code, removing it later will be much harder. In any case, locks in HDFS don't work very well, see HADOOP-656. Also, directory locks are overkill most UNIX fses don't support it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.