You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Jim Huang (JIRA)" <ji...@apache.org> on 2009/01/21 19:31:59 UTC
[jira] Created: (HADOOP-5094) Show dead nodes information in
dfsadmin -report
Show dead nodes information in dfsadmin -report
-----------------------------------------------
Key: HADOOP-5094
URL: https://issues.apache.org/jira/browse/HADOOP-5094
Project: Hadoop Core
Issue Type: New Feature
Components: dfs
Affects Versions: 0.18.2
Reporter: Jim Huang
Priority: Minor
Fix For: 0.21.0
As part of operations responsibility to bring back dead nodes, it will be good to have a quick way to obtain a list of dead data nodes.
The current way is to scrape the namenode web UI page and parse that information, but this creates load on the namenode.
In search of a less costly way, I noticed dfsadmin -report only reports data nodes with State: "In Service" and "Decommission in progress" get listed.
Asking for a cheap way to obtain a list of dead nodes.
In addition, can the following requests be reviewed for additional enhancement and changes to dfsadmin -report.
- Consistent formatting output in "Remaining raw bytes:" for the data nodes should have a space between the exact value and the parenthesized value.
Sample:
Total raw bytes: 3842232975360 (3.49 TB)
Remaining raw bytes: 146090593065(136.06 GB)
Used raw bytes: 3240864964620 (2.95 TB)
- Include the running version of Hadoop.
- What is the meaning of "Total effective bytes"?
- Display the hostname instead of the IP address for the data node (toggle option?)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5094) Show dead nodes information in
dfsadmin -report
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12673986#action_12673986 ]
Hudson commented on HADOOP-5094:
--------------------------------
Integrated in Hadoop-trunk #756 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/756/])
> Show dead nodes information in dfsadmin -report
> -----------------------------------------------
>
> Key: HADOOP-5094
> URL: https://issues.apache.org/jira/browse/HADOOP-5094
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Affects Versions: 0.18.2
> Reporter: Jim Huang
> Assignee: Jakob Homan
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: HADOOP-5094.patch, HADOOP-5094.patch, HADOOP-5094.patch
>
>
> As part of operations responsibility to bring back dead nodes, it will be good to have a quick way to obtain a list of dead data nodes.
> The current way is to scrape the namenode web UI page and parse that information, but this creates load on the namenode.
> In search of a less costly way, I noticed dfsadmin -report only reports data nodes with State: "In Service" and "Decommission in progress" get listed.
> Asking for a cheap way to obtain a list of dead nodes.
> In addition, can the following requests be reviewed for additional enhancement and changes to dfsadmin -report.
> - Consistent formatting output in "Remaining raw bytes:" for the data nodes should have a space between the exact value and the parenthesized value.
> Sample:
> Total raw bytes: 3842232975360 (3.49 TB)
> Remaining raw bytes: 146090593065(136.06 GB)
> Used raw bytes: 3240864964620 (2.95 TB)
> - Include the running version of Hadoop.
> - What is the meaning of "Total effective bytes"?
> - Display the hostname instead of the IP address for the data node (toggle option?)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5094) Show dead nodes information in
dfsadmin -report
Posted by "Jakob Homan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12668987#action_12668987 ]
Jakob Homan commented on HADOOP-5094:
-------------------------------------
At the moment there is no way to query the server version via the client protocol, and that ability seems beyond the scope of this JIRA, so that feature should probably wait. HADOOP-4368 is currently dealing with the information included in this report and it may be worthwhile to include the server version in that work. I'll open another JIRA for this.
> Show dead nodes information in dfsadmin -report
> -----------------------------------------------
>
> Key: HADOOP-5094
> URL: https://issues.apache.org/jira/browse/HADOOP-5094
> Project: Hadoop Core
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.18.2
> Reporter: Jim Huang
> Assignee: Jakob Homan
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: HADOOP-5094.patch
>
>
> As part of operations responsibility to bring back dead nodes, it will be good to have a quick way to obtain a list of dead data nodes.
> The current way is to scrape the namenode web UI page and parse that information, but this creates load on the namenode.
> In search of a less costly way, I noticed dfsadmin -report only reports data nodes with State: "In Service" and "Decommission in progress" get listed.
> Asking for a cheap way to obtain a list of dead nodes.
> In addition, can the following requests be reviewed for additional enhancement and changes to dfsadmin -report.
> - Consistent formatting output in "Remaining raw bytes:" for the data nodes should have a space between the exact value and the parenthesized value.
> Sample:
> Total raw bytes: 3842232975360 (3.49 TB)
> Remaining raw bytes: 146090593065(136.06 GB)
> Used raw bytes: 3240864964620 (2.95 TB)
> - Include the running version of Hadoop.
> - What is the meaning of "Total effective bytes"?
> - Display the hostname instead of the IP address for the data node (toggle option?)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5094) Show dead nodes information in
dfsadmin -report
Posted by "Jakob Homan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jakob Homan updated HADOOP-5094:
--------------------------------
Attachment: HADOOP-5094.patch
The patch went stale with the committing of HADOOP-4368. Nothing of substance, just some jostling over imports. Uploading a new version that applies against trunk.
> Show dead nodes information in dfsadmin -report
> -----------------------------------------------
>
> Key: HADOOP-5094
> URL: https://issues.apache.org/jira/browse/HADOOP-5094
> Project: Hadoop Core
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.18.2
> Reporter: Jim Huang
> Assignee: Jakob Homan
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: HADOOP-5094.patch, HADOOP-5094.patch, HADOOP-5094.patch
>
>
> As part of operations responsibility to bring back dead nodes, it will be good to have a quick way to obtain a list of dead data nodes.
> The current way is to scrape the namenode web UI page and parse that information, but this creates load on the namenode.
> In search of a less costly way, I noticed dfsadmin -report only reports data nodes with State: "In Service" and "Decommission in progress" get listed.
> Asking for a cheap way to obtain a list of dead nodes.
> In addition, can the following requests be reviewed for additional enhancement and changes to dfsadmin -report.
> - Consistent formatting output in "Remaining raw bytes:" for the data nodes should have a space between the exact value and the parenthesized value.
> Sample:
> Total raw bytes: 3842232975360 (3.49 TB)
> Remaining raw bytes: 146090593065(136.06 GB)
> Used raw bytes: 3240864964620 (2.95 TB)
> - Include the running version of Hadoop.
> - What is the meaning of "Total effective bytes"?
> - Display the hostname instead of the IP address for the data node (toggle option?)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5094) Show dead nodes information in
dfsadmin -report
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12668610#action_12668610 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-5094:
------------------------------------------------
>> Include the running version of Hadoop
>
> Client version and revision added to output
Should we print the server version instead of client version? The client version can be obtained by "./bin/hadoop version".
> Show dead nodes information in dfsadmin -report
> -----------------------------------------------
>
> Key: HADOOP-5094
> URL: https://issues.apache.org/jira/browse/HADOOP-5094
> Project: Hadoop Core
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.18.2
> Reporter: Jim Huang
> Assignee: Jakob Homan
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: HADOOP-5094.patch
>
>
> As part of operations responsibility to bring back dead nodes, it will be good to have a quick way to obtain a list of dead data nodes.
> The current way is to scrape the namenode web UI page and parse that information, but this creates load on the namenode.
> In search of a less costly way, I noticed dfsadmin -report only reports data nodes with State: "In Service" and "Decommission in progress" get listed.
> Asking for a cheap way to obtain a list of dead nodes.
> In addition, can the following requests be reviewed for additional enhancement and changes to dfsadmin -report.
> - Consistent formatting output in "Remaining raw bytes:" for the data nodes should have a space between the exact value and the parenthesized value.
> Sample:
> Total raw bytes: 3842232975360 (3.49 TB)
> Remaining raw bytes: 146090593065(136.06 GB)
> Used raw bytes: 3240864964620 (2.95 TB)
> - Include the running version of Hadoop.
> - What is the meaning of "Total effective bytes"?
> - Display the hostname instead of the IP address for the data node (toggle option?)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5094) Show dead nodes information in
dfsadmin -report
Posted by "Jakob Homan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jakob Homan updated HADOOP-5094:
--------------------------------
Release Note: Update the output of dfsadmin -report to delineate the live and dead nodes, as well as attempt to resolve the hostname of datanode ip addresses. Minor formatting changes.
Hadoop Flags: [Incompatible change, Reviewed] (was: [Reviewed, Incompatible change])
> Show dead nodes information in dfsadmin -report
> -----------------------------------------------
>
> Key: HADOOP-5094
> URL: https://issues.apache.org/jira/browse/HADOOP-5094
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Affects Versions: 0.18.2
> Reporter: Jim Huang
> Assignee: Jakob Homan
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: HADOOP-5094.patch, HADOOP-5094.patch, HADOOP-5094.patch
>
>
> As part of operations responsibility to bring back dead nodes, it will be good to have a quick way to obtain a list of dead data nodes.
> The current way is to scrape the namenode web UI page and parse that information, but this creates load on the namenode.
> In search of a less costly way, I noticed dfsadmin -report only reports data nodes with State: "In Service" and "Decommission in progress" get listed.
> Asking for a cheap way to obtain a list of dead nodes.
> In addition, can the following requests be reviewed for additional enhancement and changes to dfsadmin -report.
> - Consistent formatting output in "Remaining raw bytes:" for the data nodes should have a space between the exact value and the parenthesized value.
> Sample:
> Total raw bytes: 3842232975360 (3.49 TB)
> Remaining raw bytes: 146090593065(136.06 GB)
> Used raw bytes: 3240864964620 (2.95 TB)
> - Include the running version of Hadoop.
> - What is the meaning of "Total effective bytes"?
> - Display the hostname instead of the IP address for the data node (toggle option?)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5094) Show dead nodes information in
dfsadmin -report
Posted by "Jakob Homan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jakob Homan updated HADOOP-5094:
--------------------------------
Status: Patch Available (was: Open)
> Show dead nodes information in dfsadmin -report
> -----------------------------------------------
>
> Key: HADOOP-5094
> URL: https://issues.apache.org/jira/browse/HADOOP-5094
> Project: Hadoop Core
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.18.2
> Reporter: Jim Huang
> Assignee: Jakob Homan
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: HADOOP-5094.patch
>
>
> As part of operations responsibility to bring back dead nodes, it will be good to have a quick way to obtain a list of dead data nodes.
> The current way is to scrape the namenode web UI page and parse that information, but this creates load on the namenode.
> In search of a less costly way, I noticed dfsadmin -report only reports data nodes with State: "In Service" and "Decommission in progress" get listed.
> Asking for a cheap way to obtain a list of dead nodes.
> In addition, can the following requests be reviewed for additional enhancement and changes to dfsadmin -report.
> - Consistent formatting output in "Remaining raw bytes:" for the data nodes should have a space between the exact value and the parenthesized value.
> Sample:
> Total raw bytes: 3842232975360 (3.49 TB)
> Remaining raw bytes: 146090593065(136.06 GB)
> Used raw bytes: 3240864964620 (2.95 TB)
> - Include the running version of Hadoop.
> - What is the meaning of "Total effective bytes"?
> - Display the hostname instead of the IP address for the data node (toggle option?)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5094) Show dead nodes information in
dfsadmin -report
Posted by "Jakob Homan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jakob Homan updated HADOOP-5094:
--------------------------------
Attachment: HADOOP-5094.patch
This patch adds headers to the list of datanodes, separating the living from the dead:
{noformat}
-------------------------------------------------
Datanodes available: 9 (10 total, 1 dead)
Live datanodes:
Name: ipaddr:58301
Decommission Status : Normal
Configured Capacity: 974886735872 (907.93 GB)
DFS Used: 98304 (96 KB)
Non DFS Used: 163215228928 (152.01 GB)
DFS Remaining: 811671408640(755.93 GB)
DFS Used%: 0%
DFS Remaining%: 83.26%
Last contact: Wed Jan 28 23:29:32 UTC 2009
<<snip>>
Dead datanodes:
Name: ipaddr2:53655
Decommission Status : Normal
Configured Capacity: 974886735872 (907.93 GB)
DFS Used: 98304 (96 KB)
Non DFS Used: 209286926336 (194.91 GB)
DFS Remaining: 765599711232(713.02 GB)
DFS Used%: 0%
DFS Remaining%: 78.53%
Last contact: Wed Jan 28 23:17:43 UTC 2009
{noformat}
Also,
bq. Consistent formatting output in "Remaining raw bytes:"
fixed.
bq. Include the running version of Hadoop
Client version and revision added to output
bq. What is the meaning of "Total effective bytes"?
As Suresh noted, no longer included in report output
bq. Display the hostname instead of the IP address for the data node (toggle option?)
Would it be worth it have both, if the datanode is specified as an ip addr initially?
Patch passes all unit tests except known-bad HADOOP-4907. test-patch:
{noformat}
[exec] -1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] -1 tests included. The patch doesn't appear to include any new or modified tests.
[exec] Please justify why no tests are needed for this patch.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.{noformat}
No new unit tests because it's just a change to the output of the report and not easily tested.
> Show dead nodes information in dfsadmin -report
> -----------------------------------------------
>
> Key: HADOOP-5094
> URL: https://issues.apache.org/jira/browse/HADOOP-5094
> Project: Hadoop Core
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.18.2
> Reporter: Jim Huang
> Assignee: Jakob Homan
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: HADOOP-5094.patch
>
>
> As part of operations responsibility to bring back dead nodes, it will be good to have a quick way to obtain a list of dead data nodes.
> The current way is to scrape the namenode web UI page and parse that information, but this creates load on the namenode.
> In search of a less costly way, I noticed dfsadmin -report only reports data nodes with State: "In Service" and "Decommission in progress" get listed.
> Asking for a cheap way to obtain a list of dead nodes.
> In addition, can the following requests be reviewed for additional enhancement and changes to dfsadmin -report.
> - Consistent formatting output in "Remaining raw bytes:" for the data nodes should have a space between the exact value and the parenthesized value.
> Sample:
> Total raw bytes: 3842232975360 (3.49 TB)
> Remaining raw bytes: 146090593065(136.06 GB)
> Used raw bytes: 3240864964620 (2.95 TB)
> - Include the running version of Hadoop.
> - What is the meaning of "Total effective bytes"?
> - Display the hostname instead of the IP address for the data node (toggle option?)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5094) Show dead nodes information in
dfsadmin -report
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-5094:
-------------------------------------------
Issue Type: Improvement (was: New Feature)
Hadoop Flags: [Incompatible change, Reviewed] (was: [Incompatible change])
+1 patch looks good.
> Show dead nodes information in dfsadmin -report
> -----------------------------------------------
>
> Key: HADOOP-5094
> URL: https://issues.apache.org/jira/browse/HADOOP-5094
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Affects Versions: 0.18.2
> Reporter: Jim Huang
> Assignee: Jakob Homan
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: HADOOP-5094.patch, HADOOP-5094.patch, HADOOP-5094.patch
>
>
> As part of operations responsibility to bring back dead nodes, it will be good to have a quick way to obtain a list of dead data nodes.
> The current way is to scrape the namenode web UI page and parse that information, but this creates load on the namenode.
> In search of a less costly way, I noticed dfsadmin -report only reports data nodes with State: "In Service" and "Decommission in progress" get listed.
> Asking for a cheap way to obtain a list of dead nodes.
> In addition, can the following requests be reviewed for additional enhancement and changes to dfsadmin -report.
> - Consistent formatting output in "Remaining raw bytes:" for the data nodes should have a space between the exact value and the parenthesized value.
> Sample:
> Total raw bytes: 3842232975360 (3.49 TB)
> Remaining raw bytes: 146090593065(136.06 GB)
> Used raw bytes: 3240864964620 (2.95 TB)
> - Include the running version of Hadoop.
> - What is the meaning of "Total effective bytes"?
> - Display the hostname instead of the IP address for the data node (toggle option?)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5094) Show dead nodes information in
dfsadmin -report
Posted by "Jakob Homan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jakob Homan updated HADOOP-5094:
--------------------------------
Status: Open (was: Patch Available)
> Show dead nodes information in dfsadmin -report
> -----------------------------------------------
>
> Key: HADOOP-5094
> URL: https://issues.apache.org/jira/browse/HADOOP-5094
> Project: Hadoop Core
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.18.2
> Reporter: Jim Huang
> Assignee: Jakob Homan
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: HADOOP-5094.patch
>
>
> As part of operations responsibility to bring back dead nodes, it will be good to have a quick way to obtain a list of dead data nodes.
> The current way is to scrape the namenode web UI page and parse that information, but this creates load on the namenode.
> In search of a less costly way, I noticed dfsadmin -report only reports data nodes with State: "In Service" and "Decommission in progress" get listed.
> Asking for a cheap way to obtain a list of dead nodes.
> In addition, can the following requests be reviewed for additional enhancement and changes to dfsadmin -report.
> - Consistent formatting output in "Remaining raw bytes:" for the data nodes should have a space between the exact value and the parenthesized value.
> Sample:
> Total raw bytes: 3842232975360 (3.49 TB)
> Remaining raw bytes: 146090593065(136.06 GB)
> Used raw bytes: 3240864964620 (2.95 TB)
> - Include the running version of Hadoop.
> - What is the meaning of "Total effective bytes"?
> - Display the hostname instead of the IP address for the data node (toggle option?)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5094) Show dead nodes information in
dfsadmin -report
Posted by "Jakob Homan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jakob Homan updated HADOOP-5094:
--------------------------------
Description:
As part of operations responsibility to bring back dead nodes, it will be good to have a quick way to obtain a list of dead data nodes.
The current way is to scrape the namenode web UI page and parse that information, but this creates load on the namenode.
In search of a less costly way, I noticed dfsadmin -report only reports data nodes with State: "In Service" and "Decommission in progress" get listed.
Asking for a cheap way to obtain a list of dead nodes.
In addition, can the following requests be reviewed for additional enhancement and changes to dfsadmin -report.
- Consistent formatting output in "Remaining raw bytes:" for the data nodes should have a space between the exact value and the parenthesized value.
Sample:
Total raw bytes: 3842232975360 (3.49 TB)
Remaining raw bytes: 146090593065(136.06 GB)
Used raw bytes: 3240864964620 (2.95 TB)
- Include the running version of Hadoop.
- What is the meaning of "Total effective bytes"?
- Display the hostname instead of the IP address for the data node (toggle option?)
was:
As part of operations responsibility to bring back dead nodes, it will be good to have a quick way to obtain a list of dead data nodes.
The current way is to scrape the namenode web UI page and parse that information, but this creates load on the namenode.
In search of a less costly way, I noticed dfsadmin -report only reports data nodes with State: "In Service" and "Decommission in progress" get listed.
Asking for a cheap way to obtain a list of dead nodes.
In addition, can the following requests be reviewed for additional enhancement and changes to dfsadmin -report.
- Consistent formatting output in "Remaining raw bytes:" for the data nodes should have a space between the exact value and the parenthesized value.
Sample:
Total raw bytes: 3842232975360 (3.49 TB)
Remaining raw bytes: 146090593065(136.06 GB)
Used raw bytes: 3240864964620 (2.95 TB)
- Include the running version of Hadoop.
- What is the meaning of "Total effective bytes"?
- Display the hostname instead of the IP address for the data node (toggle option?)
Hadoop Flags: [Incompatible change]
> Show dead nodes information in dfsadmin -report
> -----------------------------------------------
>
> Key: HADOOP-5094
> URL: https://issues.apache.org/jira/browse/HADOOP-5094
> Project: Hadoop Core
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.18.2
> Reporter: Jim Huang
> Assignee: Jakob Homan
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: HADOOP-5094.patch
>
>
> As part of operations responsibility to bring back dead nodes, it will be good to have a quick way to obtain a list of dead data nodes.
> The current way is to scrape the namenode web UI page and parse that information, but this creates load on the namenode.
> In search of a less costly way, I noticed dfsadmin -report only reports data nodes with State: "In Service" and "Decommission in progress" get listed.
> Asking for a cheap way to obtain a list of dead nodes.
> In addition, can the following requests be reviewed for additional enhancement and changes to dfsadmin -report.
> - Consistent formatting output in "Remaining raw bytes:" for the data nodes should have a space between the exact value and the parenthesized value.
> Sample:
> Total raw bytes: 3842232975360 (3.49 TB)
> Remaining raw bytes: 146090593065(136.06 GB)
> Used raw bytes: 3240864964620 (2.95 TB)
> - Include the running version of Hadoop.
> - What is the meaning of "Total effective bytes"?
> - Display the hostname instead of the IP address for the data node (toggle option?)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5094) Show dead nodes information in
dfsadmin -report
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-5094:
-------------------------------------------
Resolution: Fixed
Hadoop Flags: [Incompatible change, Reviewed] (was: [Reviewed, Incompatible change])
Status: Resolved (was: Patch Available)
I just committed this. Thanks, Jakob!
> Show dead nodes information in dfsadmin -report
> -----------------------------------------------
>
> Key: HADOOP-5094
> URL: https://issues.apache.org/jira/browse/HADOOP-5094
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Affects Versions: 0.18.2
> Reporter: Jim Huang
> Assignee: Jakob Homan
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: HADOOP-5094.patch, HADOOP-5094.patch, HADOOP-5094.patch
>
>
> As part of operations responsibility to bring back dead nodes, it will be good to have a quick way to obtain a list of dead data nodes.
> The current way is to scrape the namenode web UI page and parse that information, but this creates load on the namenode.
> In search of a less costly way, I noticed dfsadmin -report only reports data nodes with State: "In Service" and "Decommission in progress" get listed.
> Asking for a cheap way to obtain a list of dead nodes.
> In addition, can the following requests be reviewed for additional enhancement and changes to dfsadmin -report.
> - Consistent formatting output in "Remaining raw bytes:" for the data nodes should have a space between the exact value and the parenthesized value.
> Sample:
> Total raw bytes: 3842232975360 (3.49 TB)
> Remaining raw bytes: 146090593065(136.06 GB)
> Used raw bytes: 3240864964620 (2.95 TB)
> - Include the running version of Hadoop.
> - What is the meaning of "Total effective bytes"?
> - Display the hostname instead of the IP address for the data node (toggle option?)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5094) Show dead nodes information in
dfsadmin -report
Posted by "Jakob Homan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12669832#action_12669832 ]
Jakob Homan commented on HADOOP-5094:
-------------------------------------
I created HADOOP-5159 to deal with getting the server version displayed in the report.
> Show dead nodes information in dfsadmin -report
> -----------------------------------------------
>
> Key: HADOOP-5094
> URL: https://issues.apache.org/jira/browse/HADOOP-5094
> Project: Hadoop Core
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.18.2
> Reporter: Jim Huang
> Assignee: Jakob Homan
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: HADOOP-5094.patch, HADOOP-5094.patch
>
>
> As part of operations responsibility to bring back dead nodes, it will be good to have a quick way to obtain a list of dead data nodes.
> The current way is to scrape the namenode web UI page and parse that information, but this creates load on the namenode.
> In search of a less costly way, I noticed dfsadmin -report only reports data nodes with State: "In Service" and "Decommission in progress" get listed.
> Asking for a cheap way to obtain a list of dead nodes.
> In addition, can the following requests be reviewed for additional enhancement and changes to dfsadmin -report.
> - Consistent formatting output in "Remaining raw bytes:" for the data nodes should have a space between the exact value and the parenthesized value.
> Sample:
> Total raw bytes: 3842232975360 (3.49 TB)
> Remaining raw bytes: 146090593065(136.06 GB)
> Used raw bytes: 3240864964620 (2.95 TB)
> - Include the running version of Hadoop.
> - What is the meaning of "Total effective bytes"?
> - Display the hostname instead of the IP address for the data node (toggle option?)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-5094) Show dead nodes information in
dfsadmin -report
Posted by "Jakob Homan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jakob Homan reassigned HADOOP-5094:
-----------------------------------
Assignee: Jakob Homan
> Show dead nodes information in dfsadmin -report
> -----------------------------------------------
>
> Key: HADOOP-5094
> URL: https://issues.apache.org/jira/browse/HADOOP-5094
> Project: Hadoop Core
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.18.2
> Reporter: Jim Huang
> Assignee: Jakob Homan
> Priority: Minor
> Fix For: 0.21.0
>
>
> As part of operations responsibility to bring back dead nodes, it will be good to have a quick way to obtain a list of dead data nodes.
> The current way is to scrape the namenode web UI page and parse that information, but this creates load on the namenode.
> In search of a less costly way, I noticed dfsadmin -report only reports data nodes with State: "In Service" and "Decommission in progress" get listed.
> Asking for a cheap way to obtain a list of dead nodes.
> In addition, can the following requests be reviewed for additional enhancement and changes to dfsadmin -report.
> - Consistent formatting output in "Remaining raw bytes:" for the data nodes should have a space between the exact value and the parenthesized value.
> Sample:
> Total raw bytes: 3842232975360 (3.49 TB)
> Remaining raw bytes: 146090593065(136.06 GB)
> Used raw bytes: 3240864964620 (2.95 TB)
> - Include the running version of Hadoop.
> - What is the meaning of "Total effective bytes"?
> - Display the hostname instead of the IP address for the data node (toggle option?)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5094) Show dead nodes information in
dfsadmin -report
Posted by "Jakob Homan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jakob Homan updated HADOOP-5094:
--------------------------------
Status: Patch Available (was: Open)
Submitting updated patch.
> Show dead nodes information in dfsadmin -report
> -----------------------------------------------
>
> Key: HADOOP-5094
> URL: https://issues.apache.org/jira/browse/HADOOP-5094
> Project: Hadoop Core
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.18.2
> Reporter: Jim Huang
> Assignee: Jakob Homan
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: HADOOP-5094.patch, HADOOP-5094.patch
>
>
> As part of operations responsibility to bring back dead nodes, it will be good to have a quick way to obtain a list of dead data nodes.
> The current way is to scrape the namenode web UI page and parse that information, but this creates load on the namenode.
> In search of a less costly way, I noticed dfsadmin -report only reports data nodes with State: "In Service" and "Decommission in progress" get listed.
> Asking for a cheap way to obtain a list of dead nodes.
> In addition, can the following requests be reviewed for additional enhancement and changes to dfsadmin -report.
> - Consistent formatting output in "Remaining raw bytes:" for the data nodes should have a space between the exact value and the parenthesized value.
> Sample:
> Total raw bytes: 3842232975360 (3.49 TB)
> Remaining raw bytes: 146090593065(136.06 GB)
> Used raw bytes: 3240864964620 (2.95 TB)
> - Include the running version of Hadoop.
> - What is the meaning of "Total effective bytes"?
> - Display the hostname instead of the IP address for the data node (toggle option?)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5094) Show dead nodes information in
dfsadmin -report
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12670405#action_12670405 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-5094:
------------------------------------------------
Jakob, please add release note since this is an incompatible change.
> Show dead nodes information in dfsadmin -report
> -----------------------------------------------
>
> Key: HADOOP-5094
> URL: https://issues.apache.org/jira/browse/HADOOP-5094
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Affects Versions: 0.18.2
> Reporter: Jim Huang
> Assignee: Jakob Homan
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: HADOOP-5094.patch, HADOOP-5094.patch, HADOOP-5094.patch
>
>
> As part of operations responsibility to bring back dead nodes, it will be good to have a quick way to obtain a list of dead data nodes.
> The current way is to scrape the namenode web UI page and parse that information, but this creates load on the namenode.
> In search of a less costly way, I noticed dfsadmin -report only reports data nodes with State: "In Service" and "Decommission in progress" get listed.
> Asking for a cheap way to obtain a list of dead nodes.
> In addition, can the following requests be reviewed for additional enhancement and changes to dfsadmin -report.
> - Consistent formatting output in "Remaining raw bytes:" for the data nodes should have a space between the exact value and the parenthesized value.
> Sample:
> Total raw bytes: 3842232975360 (3.49 TB)
> Remaining raw bytes: 146090593065(136.06 GB)
> Used raw bytes: 3240864964620 (2.95 TB)
> - Include the running version of Hadoop.
> - What is the meaning of "Total effective bytes"?
> - Display the hostname instead of the IP address for the data node (toggle option?)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5094) Show dead nodes information in
dfsadmin -report
Posted by "Jakob Homan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12669206#action_12669206 ]
Jakob Homan commented on HADOOP-5094:
-------------------------------------
The failing contrib tests are the known-bad Chukwa tests.
> Show dead nodes information in dfsadmin -report
> -----------------------------------------------
>
> Key: HADOOP-5094
> URL: https://issues.apache.org/jira/browse/HADOOP-5094
> Project: Hadoop Core
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.18.2
> Reporter: Jim Huang
> Assignee: Jakob Homan
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: HADOOP-5094.patch, HADOOP-5094.patch
>
>
> As part of operations responsibility to bring back dead nodes, it will be good to have a quick way to obtain a list of dead data nodes.
> The current way is to scrape the namenode web UI page and parse that information, but this creates load on the namenode.
> In search of a less costly way, I noticed dfsadmin -report only reports data nodes with State: "In Service" and "Decommission in progress" get listed.
> Asking for a cheap way to obtain a list of dead nodes.
> In addition, can the following requests be reviewed for additional enhancement and changes to dfsadmin -report.
> - Consistent formatting output in "Remaining raw bytes:" for the data nodes should have a space between the exact value and the parenthesized value.
> Sample:
> Total raw bytes: 3842232975360 (3.49 TB)
> Remaining raw bytes: 146090593065(136.06 GB)
> Used raw bytes: 3240864964620 (2.95 TB)
> - Include the running version of Hadoop.
> - What is the meaning of "Total effective bytes"?
> - Display the hostname instead of the IP address for the data node (toggle option?)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5094) Show dead nodes information in
dfsadmin -report
Posted by "Jim Huang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12668619#action_12668619 ]
Jim Huang commented on HADOOP-5094:
-----------------------------------
Please do provide the server version, so there is a quick and non-taxing way of determine what is the current running version on the namenode.
> Show dead nodes information in dfsadmin -report
> -----------------------------------------------
>
> Key: HADOOP-5094
> URL: https://issues.apache.org/jira/browse/HADOOP-5094
> Project: Hadoop Core
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.18.2
> Reporter: Jim Huang
> Assignee: Jakob Homan
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: HADOOP-5094.patch
>
>
> As part of operations responsibility to bring back dead nodes, it will be good to have a quick way to obtain a list of dead data nodes.
> The current way is to scrape the namenode web UI page and parse that information, but this creates load on the namenode.
> In search of a less costly way, I noticed dfsadmin -report only reports data nodes with State: "In Service" and "Decommission in progress" get listed.
> Asking for a cheap way to obtain a list of dead nodes.
> In addition, can the following requests be reviewed for additional enhancement and changes to dfsadmin -report.
> - Consistent formatting output in "Remaining raw bytes:" for the data nodes should have a space between the exact value and the parenthesized value.
> Sample:
> Total raw bytes: 3842232975360 (3.49 TB)
> Remaining raw bytes: 146090593065(136.06 GB)
> Used raw bytes: 3240864964620 (2.95 TB)
> - Include the running version of Hadoop.
> - What is the meaning of "Total effective bytes"?
> - Display the hostname instead of the IP address for the data node (toggle option?)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5094) Show dead nodes information in
dfsadmin -report
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12667938#action_12667938 ]
Suresh Srinivas commented on HADOOP-5094:
-----------------------------------------
Output from the command has changed (though the new output still has space before parenthesis) missing. Here is the output with the change from 4281:
Configured Capacity: 6339239936 (5.9 GB)
Present Capacity: 3782686528 (3.52 GB)
DFS Remaining: 2781669184(2.59 GB)
DFS Used: 1001017344 (954.64 MB)
DFS Used%: 26.46%
> Show dead nodes information in dfsadmin -report
> -----------------------------------------------
>
> Key: HADOOP-5094
> URL: https://issues.apache.org/jira/browse/HADOOP-5094
> Project: Hadoop Core
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.18.2
> Reporter: Jim Huang
> Assignee: Jakob Homan
> Priority: Minor
> Fix For: 0.21.0
>
>
> As part of operations responsibility to bring back dead nodes, it will be good to have a quick way to obtain a list of dead data nodes.
> The current way is to scrape the namenode web UI page and parse that information, but this creates load on the namenode.
> In search of a less costly way, I noticed dfsadmin -report only reports data nodes with State: "In Service" and "Decommission in progress" get listed.
> Asking for a cheap way to obtain a list of dead nodes.
> In addition, can the following requests be reviewed for additional enhancement and changes to dfsadmin -report.
> - Consistent formatting output in "Remaining raw bytes:" for the data nodes should have a space between the exact value and the parenthesized value.
> Sample:
> Total raw bytes: 3842232975360 (3.49 TB)
> Remaining raw bytes: 146090593065(136.06 GB)
> Used raw bytes: 3240864964620 (2.95 TB)
> - Include the running version of Hadoop.
> - What is the meaning of "Total effective bytes"?
> - Display the hostname instead of the IP address for the data node (toggle option?)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5094) Show dead nodes information in
dfsadmin -report
Posted by "Jakob Homan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jakob Homan updated HADOOP-5094:
--------------------------------
Attachment: HADOOP-5094.patch
Updated patch to add code to include the hostname in parens after ip addr, if it can be determined. If not, nothing is printed.
Good on unit tests except the usual suspect. Again, just a change in output, so no new unit test:
{noformat}
[exec] -1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] -1 tests included. The patch doesn't appear to include any new or modified tests.
[exec] Please justify why no tests are needed for this patch.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
{noformat}
> Show dead nodes information in dfsadmin -report
> -----------------------------------------------
>
> Key: HADOOP-5094
> URL: https://issues.apache.org/jira/browse/HADOOP-5094
> Project: Hadoop Core
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.18.2
> Reporter: Jim Huang
> Assignee: Jakob Homan
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: HADOOP-5094.patch, HADOOP-5094.patch
>
>
> As part of operations responsibility to bring back dead nodes, it will be good to have a quick way to obtain a list of dead data nodes.
> The current way is to scrape the namenode web UI page and parse that information, but this creates load on the namenode.
> In search of a less costly way, I noticed dfsadmin -report only reports data nodes with State: "In Service" and "Decommission in progress" get listed.
> Asking for a cheap way to obtain a list of dead nodes.
> In addition, can the following requests be reviewed for additional enhancement and changes to dfsadmin -report.
> - Consistent formatting output in "Remaining raw bytes:" for the data nodes should have a space between the exact value and the parenthesized value.
> Sample:
> Total raw bytes: 3842232975360 (3.49 TB)
> Remaining raw bytes: 146090593065(136.06 GB)
> Used raw bytes: 3240864964620 (2.95 TB)
> - Include the running version of Hadoop.
> - What is the meaning of "Total effective bytes"?
> - Display the hostname instead of the IP address for the data node (toggle option?)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5094) Show dead nodes information in
dfsadmin -report
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12669156#action_12669156 ]
Hadoop QA commented on HADOOP-5094:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12399197/HADOOP-5094.patch
against trunk revision 739416.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3778/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3778/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3778/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3778/console
This message is automatically generated.
> Show dead nodes information in dfsadmin -report
> -----------------------------------------------
>
> Key: HADOOP-5094
> URL: https://issues.apache.org/jira/browse/HADOOP-5094
> Project: Hadoop Core
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.18.2
> Reporter: Jim Huang
> Assignee: Jakob Homan
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: HADOOP-5094.patch, HADOOP-5094.patch
>
>
> As part of operations responsibility to bring back dead nodes, it will be good to have a quick way to obtain a list of dead data nodes.
> The current way is to scrape the namenode web UI page and parse that information, but this creates load on the namenode.
> In search of a less costly way, I noticed dfsadmin -report only reports data nodes with State: "In Service" and "Decommission in progress" get listed.
> Asking for a cheap way to obtain a list of dead nodes.
> In addition, can the following requests be reviewed for additional enhancement and changes to dfsadmin -report.
> - Consistent formatting output in "Remaining raw bytes:" for the data nodes should have a space between the exact value and the parenthesized value.
> Sample:
> Total raw bytes: 3842232975360 (3.49 TB)
> Remaining raw bytes: 146090593065(136.06 GB)
> Used raw bytes: 3240864964620 (2.95 TB)
> - Include the running version of Hadoop.
> - What is the meaning of "Total effective bytes"?
> - Display the hostname instead of the IP address for the data node (toggle option?)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5094) Show dead nodes information in
dfsadmin -report
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12668597#action_12668597 ]
Raghu Angadi commented on HADOOP-5094:
--------------------------------------
> Would it be worth it have both, if the datanode is specified as an ip addr initially?
I think it should be both.
> Show dead nodes information in dfsadmin -report
> -----------------------------------------------
>
> Key: HADOOP-5094
> URL: https://issues.apache.org/jira/browse/HADOOP-5094
> Project: Hadoop Core
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.18.2
> Reporter: Jim Huang
> Assignee: Jakob Homan
> Priority: Minor
> Fix For: 0.21.0
>
> Attachments: HADOOP-5094.patch
>
>
> As part of operations responsibility to bring back dead nodes, it will be good to have a quick way to obtain a list of dead data nodes.
> The current way is to scrape the namenode web UI page and parse that information, but this creates load on the namenode.
> In search of a less costly way, I noticed dfsadmin -report only reports data nodes with State: "In Service" and "Decommission in progress" get listed.
> Asking for a cheap way to obtain a list of dead nodes.
> In addition, can the following requests be reviewed for additional enhancement and changes to dfsadmin -report.
> - Consistent formatting output in "Remaining raw bytes:" for the data nodes should have a space between the exact value and the parenthesized value.
> Sample:
> Total raw bytes: 3842232975360 (3.49 TB)
> Remaining raw bytes: 146090593065(136.06 GB)
> Used raw bytes: 3240864964620 (2.95 TB)
> - Include the running version of Hadoop.
> - What is the meaning of "Total effective bytes"?
> - Display the hostname instead of the IP address for the data node (toggle option?)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.