You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Eli Collins (JIRA)" <ji...@apache.org> on 2012/10/11 01:05:04 UTC
[jira] [Created] (HDFS-4030) BlockManager excessBlocksCount and
postponedMisreplicatedBlocksCount should be AtomicLongs
Eli Collins created HDFS-4030:
---------------------------------
Summary: BlockManager excessBlocksCount and postponedMisreplicatedBlocksCount should be AtomicLongs
Key: HDFS-4030
URL: https://issues.apache.org/jira/browse/HDFS-4030
Project: Hadoop HDFS
Issue Type: Sub-task
Components: name-node
Affects Versions: 2.0.0-alpha
Reporter: Eli Collins
Assignee: Eli Collins
The BlockManager excessBlocksCount and postponedMisreplicatedBlocksCount fields are currently volatile longs which are incremented, which isn't thread safe. It looks like they're always incremented on paths that hold the NN write lock but it would be easier and less error prone for future changes if we made them AtomicLongs. The other volatile long members are just set in one thread and read in another so they're fine as is.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4030) BlockManager excessBlocksCount and
postponedMisreplicatedBlocksCount should be AtomicLongs
Posted by "Kihwal Lee (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13497798#comment-13497798 ]
Kihwal Lee commented on HDFS-4030:
----------------------------------
I think TestHDFSFileSystemContract is broken by HADOOP-9042.
> BlockManager excessBlocksCount and postponedMisreplicatedBlocksCount should be AtomicLongs
> ------------------------------------------------------------------------------------------
>
> Key: HDFS-4030
> URL: https://issues.apache.org/jira/browse/HDFS-4030
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hdfs-4030.txt, hdfs-4030.txt
>
>
> The BlockManager excessBlocksCount and postponedMisreplicatedBlocksCount fields are currently volatile longs which are incremented, which isn't thread safe. It looks like they're always incremented on paths that hold the NN write lock but it would be easier and less error prone for future changes if we made them AtomicLongs. The other volatile long members are just set in one thread and read in another so they're fine as is.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4030) BlockManager excessBlocksCount and
postponedMisreplicatedBlocksCount should be AtomicLongs
Posted by "Kihwal Lee (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13481510#comment-13481510 ]
Kihwal Lee commented on HDFS-4030:
----------------------------------
+1 (nb) The patch looks good to me.
> BlockManager excessBlocksCount and postponedMisreplicatedBlocksCount should be AtomicLongs
> ------------------------------------------------------------------------------------------
>
> Key: HDFS-4030
> URL: https://issues.apache.org/jira/browse/HDFS-4030
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hdfs-4030.txt
>
>
> The BlockManager excessBlocksCount and postponedMisreplicatedBlocksCount fields are currently volatile longs which are incremented, which isn't thread safe. It looks like they're always incremented on paths that hold the NN write lock but it would be easier and less error prone for future changes if we made them AtomicLongs. The other volatile long members are just set in one thread and read in another so they're fine as is.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4030) BlockManager excessBlocksCount and
postponedMisreplicatedBlocksCount should be AtomicLongs
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13497750#comment-13497750 ]
Hadoop QA commented on HDFS-4030:
---------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12553593/hdfs-4030.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.TestHDFSFileSystemContract
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3511//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3511//console
This message is automatically generated.
> BlockManager excessBlocksCount and postponedMisreplicatedBlocksCount should be AtomicLongs
> ------------------------------------------------------------------------------------------
>
> Key: HDFS-4030
> URL: https://issues.apache.org/jira/browse/HDFS-4030
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hdfs-4030.txt, hdfs-4030.txt
>
>
> The BlockManager excessBlocksCount and postponedMisreplicatedBlocksCount fields are currently volatile longs which are incremented, which isn't thread safe. It looks like they're always incremented on paths that hold the NN write lock but it would be easier and less error prone for future changes if we made them AtomicLongs. The other volatile long members are just set in one thread and read in another so they're fine as is.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4030) BlockManager excessBlocksCount and
postponedMisreplicatedBlocksCount should be AtomicLongs
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HDFS-4030:
------------------------------
Attachment: hdfs-4030.txt
Patch attached. Verified this fixes the findbugs warnings on version 2.5.2.
> BlockManager excessBlocksCount and postponedMisreplicatedBlocksCount should be AtomicLongs
> ------------------------------------------------------------------------------------------
>
> Key: HDFS-4030
> URL: https://issues.apache.org/jira/browse/HDFS-4030
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hdfs-4030.txt
>
>
> The BlockManager excessBlocksCount and postponedMisreplicatedBlocksCount fields are currently volatile longs which are incremented, which isn't thread safe. It looks like they're always incremented on paths that hold the NN write lock but it would be easier and less error prone for future changes if we made them AtomicLongs. The other volatile long members are just set in one thread and read in another so they're fine as is.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4030) BlockManager excessBlocksCount and
postponedMisreplicatedBlocksCount should be AtomicLongs
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HDFS-4030:
------------------------------
Attachment: hdfs-4030.txt
Updated on latest trunk.
> BlockManager excessBlocksCount and postponedMisreplicatedBlocksCount should be AtomicLongs
> ------------------------------------------------------------------------------------------
>
> Key: HDFS-4030
> URL: https://issues.apache.org/jira/browse/HDFS-4030
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hdfs-4030.txt, hdfs-4030.txt
>
>
> The BlockManager excessBlocksCount and postponedMisreplicatedBlocksCount fields are currently volatile longs which are incremented, which isn't thread safe. It looks like they're always incremented on paths that hold the NN write lock but it would be easier and less error prone for future changes if we made them AtomicLongs. The other volatile long members are just set in one thread and read in another so they're fine as is.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4030) BlockManager excessBlocksCount and
postponedMisreplicatedBlocksCount should be AtomicLongs
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13498195#comment-13498195 ]
Eli Collins commented on HDFS-4030:
-----------------------------------
Thanks for catching Kihwal, you're right, I've reverted that change.
> BlockManager excessBlocksCount and postponedMisreplicatedBlocksCount should be AtomicLongs
> ------------------------------------------------------------------------------------------
>
> Key: HDFS-4030
> URL: https://issues.apache.org/jira/browse/HDFS-4030
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hdfs-4030.txt, hdfs-4030.txt
>
>
> The BlockManager excessBlocksCount and postponedMisreplicatedBlocksCount fields are currently volatile longs which are incremented, which isn't thread safe. It looks like they're always incremented on paths that hold the NN write lock but it would be easier and less error prone for future changes if we made them AtomicLongs. The other volatile long members are just set in one thread and read in another so they're fine as is.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4030) BlockManager excessBlocksCount and
postponedMisreplicatedBlocksCount should be AtomicLongs
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473801#comment-13473801 ]
Hadoop QA commented on HDFS-4030:
---------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12548658/hdfs-4030.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.web.TestWebHdfsWithMultipleNameNodes
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3306//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3306//console
This message is automatically generated.
> BlockManager excessBlocksCount and postponedMisreplicatedBlocksCount should be AtomicLongs
> ------------------------------------------------------------------------------------------
>
> Key: HDFS-4030
> URL: https://issues.apache.org/jira/browse/HDFS-4030
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hdfs-4030.txt
>
>
> The BlockManager excessBlocksCount and postponedMisreplicatedBlocksCount fields are currently volatile longs which are incremented, which isn't thread safe. It looks like they're always incremented on paths that hold the NN write lock but it would be easier and less error prone for future changes if we made them AtomicLongs. The other volatile long members are just set in one thread and read in another so they're fine as is.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4030) BlockManager excessBlocksCount and
postponedMisreplicatedBlocksCount should be AtomicLongs
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HDFS-4030:
------------------------------
Status: Patch Available (was: Open)
> BlockManager excessBlocksCount and postponedMisreplicatedBlocksCount should be AtomicLongs
> ------------------------------------------------------------------------------------------
>
> Key: HDFS-4030
> URL: https://issues.apache.org/jira/browse/HDFS-4030
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: 2.0.0-alpha
> Reporter: Eli Collins
> Assignee: Eli Collins
> Attachments: hdfs-4030.txt
>
>
> The BlockManager excessBlocksCount and postponedMisreplicatedBlocksCount fields are currently volatile longs which are incremented, which isn't thread safe. It looks like they're always incremented on paths that hold the NN write lock but it would be easier and less error prone for future changes if we made them AtomicLongs. The other volatile long members are just set in one thread and read in another so they're fine as is.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira