You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "stack (JIRA)" <ji...@apache.org> on 2011/05/17 07:31:47 UTC
[jira] [Created] (HDFS-1948) Forward port 'hdfs-1520 lightweight
namenode operation to trigger lease reccovery'
Forward port 'hdfs-1520 lightweight namenode operation to trigger lease reccovery'
----------------------------------------------------------------------------------
Key: HDFS-1948
URL: https://issues.apache.org/jira/browse/HDFS-1948
Project: Hadoop HDFS
Issue Type: Task
Reporter: stack
This issue is about forward porting from branch-0.20-append the little namenode api that facilitates stealing of a file's lease. The forward port would be an adaption of hdfs-1520 and its companion patches, hdfs-1555 and hdfs-1554, to suit the TRUNK.
Intent is to get this fix into 0.22 time willing; i'll run a vote to get ok on getting it added to branch. HBase needs this facility.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1948) Forward port 'hdfs-1520 lightweight
namenode operation to trigger lease reccovery'
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040084#comment-13040084 ]
Hadoop QA commented on HDFS-1948:
---------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12480626/1948-v4.txt
against trunk revision 1128009.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 4 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
+1 system test framework. The patch passed system test framework compile.
Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/641//testReport/
Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/641//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/641//console
This message is automatically generated.
> Forward port 'hdfs-1520 lightweight namenode operation to trigger lease reccovery'
> ----------------------------------------------------------------------------------
>
> Key: HDFS-1948
> URL: https://issues.apache.org/jira/browse/HDFS-1948
> Project: Hadoop HDFS
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Attachments: 1948-part1.txt, 1948-v2.txt, 1948-v3.txt, 1948-v3.txt, 1948-v4.txt
>
>
> This issue is about forward porting from branch-0.20-append the little namenode api that facilitates stealing of a file's lease. The forward port would be an adaption of hdfs-1520 and its companion patches, hdfs-1555 and hdfs-1554, to suit the TRUNK.
> Intent is to get this fix into 0.22 time willing; i'll run a vote to get ok on getting it added to branch. HBase needs this facility.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1948) Forward port 'hdfs-1520 lightweight
namenode operation to trigger lease reccovery'
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HDFS-1948:
------------------------
Status: Open (was: Patch Available)
> Forward port 'hdfs-1520 lightweight namenode operation to trigger lease reccovery'
> ----------------------------------------------------------------------------------
>
> Key: HDFS-1948
> URL: https://issues.apache.org/jira/browse/HDFS-1948
> Project: Hadoop HDFS
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Attachments: 1948-part1.txt, 1948-v2.txt, 1948-v3.txt, 1948-v3.txt, 1948-v4-minus_rpc_version_change.txt, 1948-v4.22.txt, 1948-v4.txt
>
>
> This issue is about forward porting from branch-0.20-append the little namenode api that facilitates stealing of a file's lease. The forward port would be an adaption of hdfs-1520 and its companion patches, hdfs-1555 and hdfs-1554, to suit the TRUNK.
> Intent is to get this fix into 0.22 time willing; i'll run a vote to get ok on getting it added to branch. HBase needs this facility.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1948) Forward port 'hdfs-1520 lightweight
namenode operation to trigger lease reccovery'
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HDFS-1948:
------------------------
Status: Patch Available (was: Open)
> Forward port 'hdfs-1520 lightweight namenode operation to trigger lease reccovery'
> ----------------------------------------------------------------------------------
>
> Key: HDFS-1948
> URL: https://issues.apache.org/jira/browse/HDFS-1948
> Project: Hadoop HDFS
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Attachments: 1948-part1.txt, 1948-v2.txt, 1948-v3.txt, 1948-v3.txt, 1948-v4-minus_rpc_version_change.txt, 1948-v4.22.txt, 1948-v4.txt
>
>
> This issue is about forward porting from branch-0.20-append the little namenode api that facilitates stealing of a file's lease. The forward port would be an adaption of hdfs-1520 and its companion patches, hdfs-1555 and hdfs-1554, to suit the TRUNK.
> Intent is to get this fix into 0.22 time willing; i'll run a vote to get ok on getting it added to branch. HBase needs this facility.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1948) Forward port 'hdfs-1520 lightweight
namenode operation to trigger lease reccovery'
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HDFS-1948:
------------------------
Status: Open (was: Patch Available)
> Forward port 'hdfs-1520 lightweight namenode operation to trigger lease reccovery'
> ----------------------------------------------------------------------------------
>
> Key: HDFS-1948
> URL: https://issues.apache.org/jira/browse/HDFS-1948
> Project: Hadoop HDFS
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Attachments: 1948-part1.txt, 1948-v2.txt, 1948-v3.txt, 1948-v3.txt, 1948-v4.txt
>
>
> This issue is about forward porting from branch-0.20-append the little namenode api that facilitates stealing of a file's lease. The forward port would be an adaption of hdfs-1520 and its companion patches, hdfs-1555 and hdfs-1554, to suit the TRUNK.
> Intent is to get this fix into 0.22 time willing; i'll run a vote to get ok on getting it added to branch. HBase needs this facility.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1948) Forward port 'hdfs-1520 lightweight
namenode operation to trigger lease reccovery'
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040064#comment-13040064 ]
Hadoop QA commented on HDFS-1948:
---------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12480625/1948-v3.txt
against trunk revision 1128009.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 4 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/640//console
This message is automatically generated.
> Forward port 'hdfs-1520 lightweight namenode operation to trigger lease reccovery'
> ----------------------------------------------------------------------------------
>
> Key: HDFS-1948
> URL: https://issues.apache.org/jira/browse/HDFS-1948
> Project: Hadoop HDFS
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Attachments: 1948-part1.txt, 1948-v2.txt, 1948-v3.txt, 1948-v3.txt
>
>
> This issue is about forward porting from branch-0.20-append the little namenode api that facilitates stealing of a file's lease. The forward port would be an adaption of hdfs-1520 and its companion patches, hdfs-1555 and hdfs-1554, to suit the TRUNK.
> Intent is to get this fix into 0.22 time willing; i'll run a vote to get ok on getting it added to branch. HBase needs this facility.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1948) Forward port 'hdfs-1520 lightweight
namenode operation to trigger lease reccovery'
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HDFS-1948:
------------------------
Assignee: stack
Release Note: Adds method to NameNode/ClientProtocol that allows for rude revoke of lease on current lease holder
Status: Patch Available (was: Open)
> Forward port 'hdfs-1520 lightweight namenode operation to trigger lease reccovery'
> ----------------------------------------------------------------------------------
>
> Key: HDFS-1948
> URL: https://issues.apache.org/jira/browse/HDFS-1948
> Project: Hadoop HDFS
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Attachments: 1948-part1.txt, 1948-v2.txt, 1948-v3.txt, 1948-v3.txt
>
>
> This issue is about forward porting from branch-0.20-append the little namenode api that facilitates stealing of a file's lease. The forward port would be an adaption of hdfs-1520 and its companion patches, hdfs-1555 and hdfs-1554, to suit the TRUNK.
> Intent is to get this fix into 0.22 time willing; i'll run a vote to get ok on getting it added to branch. HBase needs this facility.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1948) Forward port 'hdfs-1520 lightweight
namenode operation to trigger lease reccovery'
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13045154#comment-13045154 ]
Hadoop QA commented on HDFS-1948:
---------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12481623/1948-v4-minus_rpc_version_change.txt
against trunk revision 1132715.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 4 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
+1 system test framework. The patch passed system test framework compile.
Test results: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/728//testReport/
Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/728//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/728//console
This message is automatically generated.
> Forward port 'hdfs-1520 lightweight namenode operation to trigger lease reccovery'
> ----------------------------------------------------------------------------------
>
> Key: HDFS-1948
> URL: https://issues.apache.org/jira/browse/HDFS-1948
> Project: Hadoop HDFS
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Attachments: 1948-part1.txt, 1948-v2.txt, 1948-v3.txt, 1948-v3.txt, 1948-v4-minus_rpc_version_change.txt, 1948-v4.22.txt, 1948-v4.txt
>
>
> This issue is about forward porting from branch-0.20-append the little namenode api that facilitates stealing of a file's lease. The forward port would be an adaption of hdfs-1520 and its companion patches, hdfs-1555 and hdfs-1554, to suit the TRUNK.
> Intent is to get this fix into 0.22 time willing; i'll run a vote to get ok on getting it added to branch. HBase needs this facility.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1948) Forward port 'hdfs-1520 lightweight
namenode operation to trigger lease reccovery'
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HDFS-1948:
------------------------
Attachment: 1948-part1.txt
Here is bulk of the port from branch-0.20-append of the recoverLease addition added by the following JIRAs.
HDFS-1520. Lightweight NameNode operation recoverLease to trigger lease recovery. (Hairong Kuang via dhruba)
HDFS-1555. Disallow pipelien recovery if a file is already being ease recovered (hairong)
HDFS-1554. New semantics for recoverLease (hairong)
Its not complete yet. The NameNode no longer has methods nextGenerationStamp (DatanodeProtocol) -- they are present in 0.20.x but not in TRUNK -- nor
nextGenerationStampForBlock. I'm trying to figure what the equivalents are (Pointers welcome -- thanks).
> Forward port 'hdfs-1520 lightweight namenode operation to trigger lease reccovery'
> ----------------------------------------------------------------------------------
>
> Key: HDFS-1948
> URL: https://issues.apache.org/jira/browse/HDFS-1948
> Project: Hadoop HDFS
> Issue Type: Task
> Reporter: stack
> Attachments: 1948-part1.txt
>
>
> This issue is about forward porting from branch-0.20-append the little namenode api that facilitates stealing of a file's lease. The forward port would be an adaption of hdfs-1520 and its companion patches, hdfs-1555 and hdfs-1554, to suit the TRUNK.
> Intent is to get this fix into 0.22 time willing; i'll run a vote to get ok on getting it added to branch. HBase needs this facility.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1948) Forward port 'hdfs-1520 lightweight
namenode operation to trigger lease reccovery'
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13048302#comment-13048302 ]
Hudson commented on HDFS-1948:
------------------------------
Integrated in Hadoop-Hdfs-22-branch #66 (See [https://builds.apache.org/job/Hadoop-Hdfs-22-branch/66/])
> Forward port 'hdfs-1520 lightweight namenode operation to trigger lease reccovery'
> ----------------------------------------------------------------------------------
>
> Key: HDFS-1948
> URL: https://issues.apache.org/jira/browse/HDFS-1948
> Project: Hadoop HDFS
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Fix For: 0.22.0
>
> Attachments: 1948-part1.txt, 1948-v2.txt, 1948-v3.txt, 1948-v3.txt, 1948-v4-minus_rpc_version_change.txt, 1948-v4.22.txt, 1948-v4.txt
>
>
> This issue is about forward porting from branch-0.20-append the little namenode api that facilitates stealing of a file's lease. The forward port would be an adaption of hdfs-1520 and its companion patches, hdfs-1555 and hdfs-1554, to suit the TRUNK.
> Intent is to get this fix into 0.22 time willing; i'll run a vote to get ok on getting it added to branch. HBase needs this facility.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1948) Forward port 'hdfs-1520 lightweight
namenode operation to trigger lease reccovery'
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13044196#comment-13044196 ]
Hadoop QA commented on HDFS-1948:
---------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12481434/1948-v4.22.txt
against trunk revision 1131264.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 4 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/hudson/job/PreCommit-HDFS-Build/707//console
This message is automatically generated.
> Forward port 'hdfs-1520 lightweight namenode operation to trigger lease reccovery'
> ----------------------------------------------------------------------------------
>
> Key: HDFS-1948
> URL: https://issues.apache.org/jira/browse/HDFS-1948
> Project: Hadoop HDFS
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Attachments: 1948-part1.txt, 1948-v2.txt, 1948-v3.txt, 1948-v3.txt, 1948-v4.22.txt, 1948-v4.txt
>
>
> This issue is about forward porting from branch-0.20-append the little namenode api that facilitates stealing of a file's lease. The forward port would be an adaption of hdfs-1520 and its companion patches, hdfs-1555 and hdfs-1554, to suit the TRUNK.
> Intent is to get this fix into 0.22 time willing; i'll run a vote to get ok on getting it added to branch. HBase needs this facility.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1948) Forward port 'hdfs-1520 lightweight
namenode operation to trigger lease reccovery'
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HDFS-1948:
------------------------
Attachment: 1948-v4.txt
Forgot to --no-prefix making patch
> Forward port 'hdfs-1520 lightweight namenode operation to trigger lease reccovery'
> ----------------------------------------------------------------------------------
>
> Key: HDFS-1948
> URL: https://issues.apache.org/jira/browse/HDFS-1948
> Project: Hadoop HDFS
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Attachments: 1948-part1.txt, 1948-v2.txt, 1948-v3.txt, 1948-v3.txt, 1948-v4.txt
>
>
> This issue is about forward porting from branch-0.20-append the little namenode api that facilitates stealing of a file's lease. The forward port would be an adaption of hdfs-1520 and its companion patches, hdfs-1555 and hdfs-1554, to suit the TRUNK.
> Intent is to get this fix into 0.22 time willing; i'll run a vote to get ok on getting it added to branch. HBase needs this facility.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1948) Forward port 'hdfs-1520 lightweight
namenode operation to trigger lease reccovery'
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HDFS-1948:
------------------------
Attachment: 1948-v3.txt
> Forward port 'hdfs-1520 lightweight namenode operation to trigger lease reccovery'
> ----------------------------------------------------------------------------------
>
> Key: HDFS-1948
> URL: https://issues.apache.org/jira/browse/HDFS-1948
> Project: Hadoop HDFS
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Attachments: 1948-part1.txt, 1948-v2.txt, 1948-v3.txt, 1948-v3.txt
>
>
> This issue is about forward porting from branch-0.20-append the little namenode api that facilitates stealing of a file's lease. The forward port would be an adaption of hdfs-1520 and its companion patches, hdfs-1555 and hdfs-1554, to suit the TRUNK.
> Intent is to get this fix into 0.22 time willing; i'll run a vote to get ok on getting it added to branch. HBase needs this facility.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1948) Forward port 'hdfs-1520 lightweight
namenode operation to trigger lease reccovery'
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HDFS-1948:
------------------------
Status: Patch Available (was: Open)
> Forward port 'hdfs-1520 lightweight namenode operation to trigger lease reccovery'
> ----------------------------------------------------------------------------------
>
> Key: HDFS-1948
> URL: https://issues.apache.org/jira/browse/HDFS-1948
> Project: Hadoop HDFS
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Attachments: 1948-part1.txt, 1948-v2.txt, 1948-v3.txt, 1948-v3.txt, 1948-v4.txt
>
>
> This issue is about forward porting from branch-0.20-append the little namenode api that facilitates stealing of a file's lease. The forward port would be an adaption of hdfs-1520 and its companion patches, hdfs-1555 and hdfs-1554, to suit the TRUNK.
> Intent is to get this fix into 0.22 time willing; i'll run a vote to get ok on getting it added to branch. HBase needs this facility.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1948) Forward port 'hdfs-1520 lightweight
namenode operation to trigger lease reccovery'
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13045279#comment-13045279 ]
Hairong Kuang commented on HDFS-1948:
-------------------------------------
+1 lgtm. Thanks, Stack!
> Forward port 'hdfs-1520 lightweight namenode operation to trigger lease reccovery'
> ----------------------------------------------------------------------------------
>
> Key: HDFS-1948
> URL: https://issues.apache.org/jira/browse/HDFS-1948
> Project: Hadoop HDFS
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Attachments: 1948-part1.txt, 1948-v2.txt, 1948-v3.txt, 1948-v3.txt, 1948-v4-minus_rpc_version_change.txt, 1948-v4.22.txt, 1948-v4.txt
>
>
> This issue is about forward porting from branch-0.20-append the little namenode api that facilitates stealing of a file's lease. The forward port would be an adaption of hdfs-1520 and its companion patches, hdfs-1555 and hdfs-1554, to suit the TRUNK.
> Intent is to get this fix into 0.22 time willing; i'll run a vote to get ok on getting it added to branch. HBase needs this facility.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1948) Forward port 'hdfs-1520 lightweight
namenode operation to trigger lease reccovery'
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13044979#comment-13044979 ]
Hairong Kuang commented on HDFS-1948:
-------------------------------------
Both patches look good! Stack, thanks for your time porting this feature.
For the trunk, we already moved away from the version based RPC compatibility. Instead we take the method name base approach. Check HADOOP-6904. So there is no need to bump the version number. Instead dfsclient should use the method name to see if the server supports recoverLease or not.
> Forward port 'hdfs-1520 lightweight namenode operation to trigger lease reccovery'
> ----------------------------------------------------------------------------------
>
> Key: HDFS-1948
> URL: https://issues.apache.org/jira/browse/HDFS-1948
> Project: Hadoop HDFS
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Attachments: 1948-part1.txt, 1948-v2.txt, 1948-v3.txt, 1948-v3.txt, 1948-v4.22.txt, 1948-v4.txt
>
>
> This issue is about forward porting from branch-0.20-append the little namenode api that facilitates stealing of a file's lease. The forward port would be an adaption of hdfs-1520 and its companion patches, hdfs-1555 and hdfs-1554, to suit the TRUNK.
> Intent is to get this fix into 0.22 time willing; i'll run a vote to get ok on getting it added to branch. HBase needs this facility.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1948) Forward port 'hdfs-1520 lightweight
namenode operation to trigger lease reccovery'
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13044195#comment-13044195 ]
stack commented on HDFS-1948:
-----------------------------
I've been testing w/ this 0.22 patch and the lease grabbing seems to be working properly (We have explicit unit test in hbase for this -- it passes -- but did some cluster load killing of servers to test recovery and seems to do the right thing).
> Forward port 'hdfs-1520 lightweight namenode operation to trigger lease reccovery'
> ----------------------------------------------------------------------------------
>
> Key: HDFS-1948
> URL: https://issues.apache.org/jira/browse/HDFS-1948
> Project: Hadoop HDFS
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Attachments: 1948-part1.txt, 1948-v2.txt, 1948-v3.txt, 1948-v3.txt, 1948-v4.22.txt, 1948-v4.txt
>
>
> This issue is about forward porting from branch-0.20-append the little namenode api that facilitates stealing of a file's lease. The forward port would be an adaption of hdfs-1520 and its companion patches, hdfs-1555 and hdfs-1554, to suit the TRUNK.
> Intent is to get this fix into 0.22 time willing; i'll run a vote to get ok on getting it added to branch. HBase needs this facility.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1948) Forward port 'hdfs-1520 lightweight
namenode operation to trigger lease reccovery'
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HDFS-1948:
------------------------
Attachment: 1948-v3.txt
Tests pass now (Had to figure the new-fangled user faking).
> Forward port 'hdfs-1520 lightweight namenode operation to trigger lease reccovery'
> ----------------------------------------------------------------------------------
>
> Key: HDFS-1948
> URL: https://issues.apache.org/jira/browse/HDFS-1948
> Project: Hadoop HDFS
> Issue Type: Task
> Reporter: stack
> Attachments: 1948-part1.txt, 1948-v2.txt, 1948-v3.txt
>
>
> This issue is about forward porting from branch-0.20-append the little namenode api that facilitates stealing of a file's lease. The forward port would be an adaption of hdfs-1520 and its companion patches, hdfs-1555 and hdfs-1554, to suit the TRUNK.
> Intent is to get this fix into 0.22 time willing; i'll run a vote to get ok on getting it added to branch. HBase needs this facility.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1948) Forward port 'hdfs-1520 lightweight
namenode operation to trigger lease reccovery'
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13045067#comment-13045067 ]
Hairong Kuang commented on HDFS-1948:
-------------------------------------
Yes, please update the patch for the trunk.
> Forward port 'hdfs-1520 lightweight namenode operation to trigger lease reccovery'
> ----------------------------------------------------------------------------------
>
> Key: HDFS-1948
> URL: https://issues.apache.org/jira/browse/HDFS-1948
> Project: Hadoop HDFS
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Attachments: 1948-part1.txt, 1948-v2.txt, 1948-v3.txt, 1948-v3.txt, 1948-v4.22.txt, 1948-v4.txt
>
>
> This issue is about forward porting from branch-0.20-append the little namenode api that facilitates stealing of a file's lease. The forward port would be an adaption of hdfs-1520 and its companion patches, hdfs-1555 and hdfs-1554, to suit the TRUNK.
> Intent is to get this fix into 0.22 time willing; i'll run a vote to get ok on getting it added to branch. HBase needs this facility.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1948) Forward port 'hdfs-1520 lightweight
namenode operation to trigger lease reccovery'
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HDFS-1948:
------------------------
Attachment: 1948-v4.22.txt
Here is a version for branch 0.22. Two minor changes only -- the rpc version and how to get to leases.
> Forward port 'hdfs-1520 lightweight namenode operation to trigger lease reccovery'
> ----------------------------------------------------------------------------------
>
> Key: HDFS-1948
> URL: https://issues.apache.org/jira/browse/HDFS-1948
> Project: Hadoop HDFS
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Attachments: 1948-part1.txt, 1948-v2.txt, 1948-v3.txt, 1948-v3.txt, 1948-v4.22.txt, 1948-v4.txt
>
>
> This issue is about forward porting from branch-0.20-append the little namenode api that facilitates stealing of a file's lease. The forward port would be an adaption of hdfs-1520 and its companion patches, hdfs-1555 and hdfs-1554, to suit the TRUNK.
> Intent is to get this fix into 0.22 time willing; i'll run a vote to get ok on getting it added to branch. HBase needs this facility.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1948) Forward port 'hdfs-1520 lightweight
namenode operation to trigger lease reccovery'
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13044986#comment-13044986 ]
stack commented on HDFS-1948:
-----------------------------
@Hairong Thanks for the review. OK with you if I commit the patches (I'll omit the bumping of rpc on commit)?
> Forward port 'hdfs-1520 lightweight namenode operation to trigger lease reccovery'
> ----------------------------------------------------------------------------------
>
> Key: HDFS-1948
> URL: https://issues.apache.org/jira/browse/HDFS-1948
> Project: Hadoop HDFS
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Attachments: 1948-part1.txt, 1948-v2.txt, 1948-v3.txt, 1948-v3.txt, 1948-v4.22.txt, 1948-v4.txt
>
>
> This issue is about forward porting from branch-0.20-append the little namenode api that facilitates stealing of a file's lease. The forward port would be an adaption of hdfs-1520 and its companion patches, hdfs-1555 and hdfs-1554, to suit the TRUNK.
> Intent is to get this fix into 0.22 time willing; i'll run a vote to get ok on getting it added to branch. HBase needs this facility.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1948) Forward port 'hdfs-1520 lightweight
namenode operation to trigger lease reccovery'
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13050108#comment-13050108 ]
Hudson commented on HDFS-1948:
------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #746 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/746/])
> Forward port 'hdfs-1520 lightweight namenode operation to trigger lease reccovery'
> ----------------------------------------------------------------------------------
>
> Key: HDFS-1948
> URL: https://issues.apache.org/jira/browse/HDFS-1948
> Project: Hadoop HDFS
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Fix For: 0.22.0
>
> Attachments: 1948-part1.txt, 1948-v2.txt, 1948-v3.txt, 1948-v3.txt, 1948-v4-minus_rpc_version_change.txt, 1948-v4.22.txt, 1948-v4.txt
>
>
> This issue is about forward porting from branch-0.20-append the little namenode api that facilitates stealing of a file's lease. The forward port would be an adaption of hdfs-1520 and its companion patches, hdfs-1555 and hdfs-1554, to suit the TRUNK.
> Intent is to get this fix into 0.22 time willing; i'll run a vote to get ok on getting it added to branch. HBase needs this facility.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1948) Forward port 'hdfs-1520 lightweight
namenode operation to trigger lease reccovery'
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13045157#comment-13045157 ]
stack commented on HDFS-1948:
-----------------------------
@Hairong I uploaded patch and above is what jenkins thinks of this version of the patch; its the same as v4 only minus the rpc version setting. You OK w/ my going ahead and committing this? (Thanks).
> Forward port 'hdfs-1520 lightweight namenode operation to trigger lease reccovery'
> ----------------------------------------------------------------------------------
>
> Key: HDFS-1948
> URL: https://issues.apache.org/jira/browse/HDFS-1948
> Project: Hadoop HDFS
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Attachments: 1948-part1.txt, 1948-v2.txt, 1948-v3.txt, 1948-v3.txt, 1948-v4-minus_rpc_version_change.txt, 1948-v4.22.txt, 1948-v4.txt
>
>
> This issue is about forward porting from branch-0.20-append the little namenode api that facilitates stealing of a file's lease. The forward port would be an adaption of hdfs-1520 and its companion patches, hdfs-1555 and hdfs-1554, to suit the TRUNK.
> Intent is to get this fix into 0.22 time willing; i'll run a vote to get ok on getting it added to branch. HBase needs this facility.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1948) Forward port 'hdfs-1520 lightweight
namenode operation to trigger lease reccovery'
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HDFS-1948:
------------------------
Resolution: Fixed
Fix Version/s: 0.22.0
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
Applied patch to TRUNK and to branch-0.22 (HBase won't work on hadoop 0.22.0 if this patch is not present). Thanks for help and review Hairong.
> Forward port 'hdfs-1520 lightweight namenode operation to trigger lease reccovery'
> ----------------------------------------------------------------------------------
>
> Key: HDFS-1948
> URL: https://issues.apache.org/jira/browse/HDFS-1948
> Project: Hadoop HDFS
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Fix For: 0.22.0
>
> Attachments: 1948-part1.txt, 1948-v2.txt, 1948-v3.txt, 1948-v3.txt, 1948-v4-minus_rpc_version_change.txt, 1948-v4.22.txt, 1948-v4.txt
>
>
> This issue is about forward porting from branch-0.20-append the little namenode api that facilitates stealing of a file's lease. The forward port would be an adaption of hdfs-1520 and its companion patches, hdfs-1555 and hdfs-1554, to suit the TRUNK.
> Intent is to get this fix into 0.22 time willing; i'll run a vote to get ok on getting it added to branch. HBase needs this facility.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1948) Forward port 'hdfs-1520 lightweight
namenode operation to trigger lease reccovery'
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13051031#comment-13051031 ]
Hudson commented on HDFS-1948:
------------------------------
Integrated in Hadoop-Hdfs-trunk #699 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/699/])
> Forward port 'hdfs-1520 lightweight namenode operation to trigger lease reccovery'
> ----------------------------------------------------------------------------------
>
> Key: HDFS-1948
> URL: https://issues.apache.org/jira/browse/HDFS-1948
> Project: Hadoop HDFS
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Fix For: 0.22.0
>
> Attachments: 1948-part1.txt, 1948-v2.txt, 1948-v3.txt, 1948-v3.txt, 1948-v4-minus_rpc_version_change.txt, 1948-v4.22.txt, 1948-v4.txt
>
>
> This issue is about forward porting from branch-0.20-append the little namenode api that facilitates stealing of a file's lease. The forward port would be an adaption of hdfs-1520 and its companion patches, hdfs-1555 and hdfs-1554, to suit the TRUNK.
> Intent is to get this fix into 0.22 time willing; i'll run a vote to get ok on getting it added to branch. HBase needs this facility.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1948) Forward port 'hdfs-1520 lightweight
namenode operation to trigger lease reccovery'
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HDFS-1948:
------------------------
Attachment: 1948-v4-minus_rpc_version_change.txt
v4 of the patch for TRUNK minus rpc version increment.
> Forward port 'hdfs-1520 lightweight namenode operation to trigger lease reccovery'
> ----------------------------------------------------------------------------------
>
> Key: HDFS-1948
> URL: https://issues.apache.org/jira/browse/HDFS-1948
> Project: Hadoop HDFS
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Attachments: 1948-part1.txt, 1948-v2.txt, 1948-v3.txt, 1948-v3.txt, 1948-v4-minus_rpc_version_change.txt, 1948-v4.22.txt, 1948-v4.txt
>
>
> This issue is about forward porting from branch-0.20-append the little namenode api that facilitates stealing of a file's lease. The forward port would be an adaption of hdfs-1520 and its companion patches, hdfs-1555 and hdfs-1554, to suit the TRUNK.
> Intent is to get this fix into 0.22 time willing; i'll run a vote to get ok on getting it added to branch. HBase needs this facility.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1948) Forward port 'hdfs-1520 lightweight
namenode operation to trigger lease reccovery'
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HDFS-1948:
------------------------
Attachment: 1948-v2.txt
Here is full patch. Wrangling my test trying to get it to pass but putting this up meantime.
> Forward port 'hdfs-1520 lightweight namenode operation to trigger lease reccovery'
> ----------------------------------------------------------------------------------
>
> Key: HDFS-1948
> URL: https://issues.apache.org/jira/browse/HDFS-1948
> Project: Hadoop HDFS
> Issue Type: Task
> Reporter: stack
> Attachments: 1948-part1.txt, 1948-v2.txt
>
>
> This issue is about forward porting from branch-0.20-append the little namenode api that facilitates stealing of a file's lease. The forward port would be an adaption of hdfs-1520 and its companion patches, hdfs-1555 and hdfs-1554, to suit the TRUNK.
> Intent is to get this fix into 0.22 time willing; i'll run a vote to get ok on getting it added to branch. HBase needs this facility.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira