[jira] Updated: (HADOOP-5094) Show dead nodes information in dfsadmin -report

["Jakob Homan (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jakob Homan updated HADOOP-5094:
--------------------------------

    Attachment: HADOOP-5094.patch

This patch adds headers to the list of datanodes, separating the living from the dead:
{noformat}
-------------------------------------------------
Datanodes available: 9 (10 total, 1 dead)

Live datanodes:
Name: ipaddr:58301
Decommission Status : Normal
Configured Capacity: 974886735872 (907.93 GB)
DFS Used: 98304 (96 KB)
Non DFS Used: 163215228928 (152.01 GB)
DFS Remaining: 811671408640(755.93 GB)
DFS Used%: 0%
DFS Remaining%: 83.26%
Last contact: Wed Jan 28 23:29:32 UTC 2009
<<snip>>

Dead datanodes:
Name: ipaddr2:53655
Decommission Status : Normal
Configured Capacity: 974886735872 (907.93 GB)
DFS Used: 98304 (96 KB)
Non DFS Used: 209286926336 (194.91 GB)
DFS Remaining: 765599711232(713.02 GB)
DFS Used%: 0%
DFS Remaining%: 78.53%
Last contact: Wed Jan 28 23:17:43 UTC 2009
{noformat}

Also,
bq. Consistent formatting output in "Remaining raw bytes:"
fixed.

bq. Include the running version of Hadoop
Client version and revision added to output

bq. What is the meaning of "Total effective bytes"?
As Suresh noted, no longer included in report output

bq. Display the hostname instead of the IP address for the data node (toggle option?)
Would it be worth it have both, if the datanode is specified as an ip addr initially?  

Patch passes all unit tests except known-bad HADOOP-4907.  test-patch:
{noformat}
     [exec] -1 overall.  
     [exec] 
     [exec]     +1 @author.  The patch does not contain any @author tags.
     [exec] 
     [exec]     -1 tests included.  The patch doesn't appear to include any new or modified tests.
     [exec]                         Please justify why no tests are needed for this patch.
     [exec] 
     [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
     [exec] 
     [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
     [exec] 
     [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
     [exec] 
     [exec]     +1 Eclipse classpath. The patch retains Eclipse classpath integrity.{noformat}
No new unit tests because it's just a change to the output of the report and not easily tested.  



> Show dead nodes information in dfsadmin -report
> -----------------------------------------------
>
>                 Key: HADOOP-5094
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5094
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: dfs
>    Affects Versions: 0.18.2
>            Reporter: Jim Huang
>            Assignee: Jakob Homan
>            Priority: Minor
>             Fix For: 0.21.0
>
>         Attachments: HADOOP-5094.patch
>
>
> As part of operations responsibility to bring back dead nodes, it will be good to have a quick way to obtain a list of dead data nodes.  
> The current way is to scrape the namenode web UI page and parse that information, but this creates load on the namenode.   
> In search of a less costly way, I noticed dfsadmin -report only reports data nodes with State: "In Service" and "Decommission in progress" get listed.
> Asking for a cheap way to obtain a list of dead nodes.  
> In addition, can the following requests be reviewed for additional enhancement and changes to dfsadmin -report.
> - Consistent formatting output in "Remaining raw bytes:" for the data nodes should have a space between the exact value and the parenthesized value.
> Sample:
> Total raw bytes: 3842232975360 (3.49 TB)
> Remaining raw bytes: 146090593065(136.06 GB)
> Used raw bytes: 3240864964620 (2.95 TB)
> - Include the running version of Hadoop.  
> - What is the meaning of "Total effective bytes"?
> - Display the hostname instead of the IP address for the data node (toggle option?)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.