You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by Dhruba Borthakur <dh...@gmail.com> on 2010/12/28 20:12:46 UTC
Review Request: Make exiting safemode a lot faster.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/196/
-----------------------------------------------------------
Review request for hadoop-hdfs.
Summary
-------
Make exiting safemode a lot faster.
This addresses bug HDFS-1391.
https://issues.apache.org/jira/browse/HDFS-1391
Diffs
-----
http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/server/namenode/BlockManager.java 1053405
http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java 1053405
http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestOverReplicatedBlocks.java 1053405
Diff: https://reviews.apache.org/r/196/diff
Testing
-------
Running on our biggest production cluster of 3000 nodes since Dec 13th
Thanks,
Dhruba
Re: Review Request: Make exiting safemode a lot faster.
Posted by Todd Lipcon <to...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/196/#review83
-----------------------------------------------------------
http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/server/namenode/BlockManager.java
<https://reviews.apache.org/r/196/#comment146>
maybe assert namesystem.hasWriteLock() here? I think it's all correct, but provides good documentation.
http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/server/namenode/BlockManager.java
<https://reviews.apache.org/r/196/#comment144>
should probably note that these are "out" parameters
http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/server/namenode/BlockManager.java
<https://reviews.apache.org/r/196/#comment147>
excessReplicateTmp.isEmpty()?
http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/server/namenode/BlockManager.java
<https://reviews.apache.org/r/196/#comment148>
!originalDatanodes.isEmpty()
http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/server/namenode/BlockManager.java
<https://reviews.apache.org/r/196/#comment149>
can you keep this as a separate addToExcessReplicate function?
http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/server/namenode/BlockManager.java
<https://reviews.apache.org/r/196/#comment151>
add back the isDebugEnabled guard?
http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/server/namenode/BlockManager.java
<https://reviews.apache.org/r/196/#comment150>
this is just moved code, but this comment should say "instructions to the datanode" right?
http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/server/namenode/BlockManager.java
<https://reviews.apache.org/r/196/#comment152>
this is already logged inside addToInvalidates. Perhaps pass third argument false to that function so it doesn't log, if you want the specialized message?
http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
<https://reviews.apache.org/r/196/#comment145>
update javadoc to explain excessReplicateMapTmp is where the results end up
Also maybe worth noting here and in the replicator interface that 'inode' here might get modified by another thread during execution?
- Todd
On 2010-12-28 11:12:46, Dhruba Borthakur wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/196/
> -----------------------------------------------------------
>
> (Updated 2010-12-28 11:12:46)
>
>
> Review request for hadoop-hdfs.
>
>
> Summary
> -------
>
> Make exiting safemode a lot faster.
>
>
> This addresses bug HDFS-1391.
> https://issues.apache.org/jira/browse/HDFS-1391
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/server/namenode/BlockManager.java 1053405
> http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java 1053405
> http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestOverReplicatedBlocks.java 1053405
>
> Diff: https://reviews.apache.org/r/196/diff
>
>
> Testing
> -------
>
> Running on our biggest production cluster of 3000 nodes since Dec 13th
>
>
> Thanks,
>
> Dhruba
>
>
Re: Review Request: Make exiting safemode a lot faster.
Posted by Todd Lipcon <to...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/196/#review149
-----------------------------------------------------------
I think you attached the wrong patch? This seems like the patch for HDFS-1508?
- Todd
On 2011-01-23 20:24:11, Dhruba Borthakur wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/196/
> -----------------------------------------------------------
>
> (Updated 2011-01-23 20:24:11)
>
>
> Review request for hadoop-hdfs.
>
>
> Summary
> -------
>
> Make exiting safemode a lot faster.
>
>
> This addresses bug HDFS-1391.
> https://issues.apache.org/jira/browse/HDFS-1391
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/DFSClient.java 1062627
> http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/DistributedFileSystem.java 1062627
> http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java 1062627
> http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java 1062627
> http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java 1062627
> http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java 1062627
> http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/test/hdfs/org/apache/hadoop/hdfs/UpgradeUtilities.java 1062627
> http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestEditLogRace.java 1062627
> http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestParallelImageWrite.java 1062627
> http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestSaveNamespace.java 1062627
> http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestStartup.java 1062627
> http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/test/hdfs/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewer.java 1062627
>
> Diff: https://reviews.apache.org/r/196/diff
>
>
> Testing
> -------
>
> Running on our biggest production cluster of 3000 nodes since Dec 13th
>
>
> Thanks,
>
> Dhruba
>
>
Re: Review Request: Make exiting safemode a lot faster.
Posted by Dhruba Borthakur <dh...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/196/
-----------------------------------------------------------
(Updated 2011-01-23 20:24:11.634622)
Review request for hadoop-hdfs.
Changes
-------
Addressed review comments.
Summary
-------
Make exiting safemode a lot faster.
This addresses bug HDFS-1391.
https://issues.apache.org/jira/browse/HDFS-1391
Diffs (updated)
-----
http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/DFSClient.java 1062627
http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/DistributedFileSystem.java 1062627
http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java 1062627
http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java 1062627
http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java 1062627
http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java 1062627
http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/test/hdfs/org/apache/hadoop/hdfs/UpgradeUtilities.java 1062627
http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestEditLogRace.java 1062627
http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestParallelImageWrite.java 1062627
http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestSaveNamespace.java 1062627
http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestStartup.java 1062627
http://svn.apache.org/repos/asf/hadoop/hdfs/trunk/src/test/hdfs/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewer.java 1062627
Diff: https://reviews.apache.org/r/196/diff
Testing
-------
Running on our biggest production cluster of 3000 nodes since Dec 13th
Thanks,
Dhruba