You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "dhruba borthakur (JIRA)" <ji...@apache.org> on 2007/03/21 23:39:32 UTC

[jira] Created: (HADOOP-1138) Datanodes that are dead for a long long time should not show up in the UI

Datanodes that are dead for a long long time should not show up in the UI
-------------------------------------------------------------------------

                 Key: HADOOP-1138
                 URL: https://issues.apache.org/jira/browse/HADOOP-1138
             Project: Hadoop
          Issue Type: Bug
          Components: dfs
            Reporter: dhruba borthakur
         Assigned To: dhruba borthakur


Proposal 1:
If a include files is used, then show all nodes (dead/alive) that are listed in the includes file. If there isn't an include file, then display only nodes that have pinged this instance of the namenode.

Proposal2:
A config variable specifies the time duration. The namenode, on a restart, purges all datanodes that have not pinged for that time duration. The default value of this config variable can be 1 week. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-1138) Datanodes that are dead for a long long time should not show up in the UI

Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Raghu Angadi updated HADOOP-1138:
---------------------------------

    Status: Open  (was: Patch Available)

> Datanodes that are dead for a long long time should not show up in the UI
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-1138
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1138
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: Raghu Angadi
>             Fix For: 0.15.0
>
>         Attachments: HADOOP-1138.patch, HADOOP-1138.patch
>
>
> Proposal 1:
> If a include files is used, then show all nodes (dead/alive) that are listed in the includes file. If there isn't an include file, then display only nodes that have pinged this instance of the namenode.
> Proposal2:
> A config variable specifies the time duration. The namenode, on a restart, purges all datanodes that have not pinged for that time duration. The default value of this config variable can be 1 week. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-1138) Datanodes that are dead for a long long time should not show up in the UI

Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Raghu Angadi updated HADOOP-1138:
---------------------------------

    Attachment: HADOOP-1138.patch

New patch attached.

bq. 1. FSnamesystem.getDatanodeListForReport excludes nodes that are listed in dfs.hosts.exclude. Maybe a better option wold be to show them with a status of "Excluded". Currently, it shows "Decommisioned" or "In Service".

Currently there is no state shown for deadnodes. Note that this method looks at dfs.hosts.exclude only for datanodes that are considered dead.

bq. 2. The comment in FSnamesystem.getDatanodeListForReport talks about "dfs.report.datanode.timeout.day" but it should be "dfs.report.datanode.timeout.hours".

Done. Good catch. You actually read the comments!

bq. 3. Maybe a unit test case that tests this functionality would be really nice.
This is a very non-consequential functionality. It only affects Namenode front page and 'dfsadmin -report'. Let me know if we really need to add a unit test. I did test it. 

> Datanodes that are dead for a long long time should not show up in the UI
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-1138
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1138
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: Raghu Angadi
>             Fix For: 0.15.0
>
>         Attachments: HADOOP-1138.patch, HADOOP-1138.patch
>
>
> Proposal 1:
> If a include files is used, then show all nodes (dead/alive) that are listed in the includes file. If there isn't an include file, then display only nodes that have pinged this instance of the namenode.
> Proposal2:
> A config variable specifies the time duration. The namenode, on a restart, purges all datanodes that have not pinged for that time duration. The default value of this config variable can be 1 week. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-1138) Datanodes that are dead for a long long time should not show up in the UI

Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12521935 ] 

Raghu Angadi commented on HADOOP-1138:
--------------------------------------

There are two related issues here :
 # Datanodes displayed in webui and "dfs -report"
 # Policy regd storing all the known storage ids and datanodes persistently in Namenode. 

I will file a different jira for the second one.


> Datanodes that are dead for a long long time should not show up in the UI
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-1138
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1138
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: Raghu Angadi
>             Fix For: 0.15.0
>
>
> Proposal 1:
> If a include files is used, then show all nodes (dead/alive) that are listed in the includes file. If there isn't an include file, then display only nodes that have pinged this instance of the namenode.
> Proposal2:
> A config variable specifies the time duration. The namenode, on a restart, purges all datanodes that have not pinged for that time duration. The default value of this config variable can be 1 week. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-1138) Datanodes that are dead for a long long time should not show up in the UI

Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12525226 ] 

Raghu Angadi commented on HADOOP-1138:
--------------------------------------

Also this will conflict with HADOOP-1838, if it is committed earlier.

> Datanodes that are dead for a long long time should not show up in the UI
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-1138
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1138
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: Raghu Angadi
>             Fix For: 0.15.0
>
>         Attachments: HADOOP-1138.patch, HADOOP-1138.patch, HADOOP-1138.patch
>
>
> Proposal 1:
> If a include files is used, then show all nodes (dead/alive) that are listed in the includes file. If there isn't an include file, then display only nodes that have pinged this instance of the namenode.
> Proposal2:
> A config variable specifies the time duration. The namenode, on a restart, purges all datanodes that have not pinged for that time duration. The default value of this config variable can be 1 week. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Resolved: (HADOOP-1138) Datanodes that are dead for a long long time should not show up in the UI

Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

dhruba borthakur resolved HADOOP-1138.
--------------------------------------

    Resolution: Won't Fix

HADOOP-1933 has follow-up discussions.

> Datanodes that are dead for a long long time should not show up in the UI
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-1138
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1138
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: Raghu Angadi
>             Fix For: 0.15.0
>
>         Attachments: HADOOP-1138.patch, HADOOP-1138.patch, HADOOP-1138.patch
>
>
> Proposal 1:
> If a include files is used, then show all nodes (dead/alive) that are listed in the includes file. If there isn't an include file, then display only nodes that have pinged this instance of the namenode.
> Proposal2:
> A config variable specifies the time duration. The namenode, on a restart, purges all datanodes that have not pinged for that time duration. The default value of this config variable can be 1 week. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-1138) Datanodes that are dead for a long long time should not show up in the UI

Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Raghu Angadi updated HADOOP-1138:
---------------------------------

    Attachment: HADOOP-1138.patch


Earlier patch did not show the nodes that were listed in exlude file (dfs.hosts.exlude). But this broke TestDecommission. The current patch treats these nodes just like the normal dead nodes.

> Datanodes that are dead for a long long time should not show up in the UI
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-1138
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1138
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: Raghu Angadi
>             Fix For: 0.15.0
>
>         Attachments: HADOOP-1138.patch, HADOOP-1138.patch, HADOOP-1138.patch
>
>
> Proposal 1:
> If a include files is used, then show all nodes (dead/alive) that are listed in the includes file. If there isn't an include file, then display only nodes that have pinged this instance of the namenode.
> Proposal2:
> A config variable specifies the time duration. The namenode, on a restart, purges all datanodes that have not pinged for that time duration. The default value of this config variable can be 1 week. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Issue Comment Edited: (HADOOP-1138) Datanodes that are dead for a long long time should not show up in the UI

Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12525330 ] 

rangadi edited comment on HADOOP-1138 at 9/6/07 12:08 AM:
---------------------------------------------------------------

This is not considered necessary after HADOOP-1762.

      was (Author: rangadi):
    This is not considered necessary after HADOO-1762.
  
> Datanodes that are dead for a long long time should not show up in the UI
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-1138
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1138
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: Raghu Angadi
>             Fix For: 0.15.0
>
>         Attachments: HADOOP-1138.patch, HADOOP-1138.patch, HADOOP-1138.patch
>
>
> Proposal 1:
> If a include files is used, then show all nodes (dead/alive) that are listed in the includes file. If there isn't an include file, then display only nodes that have pinged this instance of the namenode.
> Proposal2:
> A config variable specifies the time duration. The namenode, on a restart, purges all datanodes that have not pinged for that time duration. The default value of this config variable can be 1 week. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-1138) Datanodes that are dead for a long long time should not show up in the UI

Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Raghu Angadi updated HADOOP-1138:
---------------------------------

    Status: Patch Available  (was: Open)

> Datanodes that are dead for a long long time should not show up in the UI
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-1138
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1138
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: Raghu Angadi
>             Fix For: 0.15.0
>
>         Attachments: HADOOP-1138.patch, HADOOP-1138.patch, HADOOP-1138.patch
>
>
> Proposal 1:
> If a include files is used, then show all nodes (dead/alive) that are listed in the includes file. If there isn't an include file, then display only nodes that have pinged this instance of the namenode.
> Proposal2:
> A config variable specifies the time duration. The namenode, on a restart, purges all datanodes that have not pinged for that time duration. The default value of this config variable can be 1 week. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-1138) Datanodes that are dead for a long long time should not show up in the UI

Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Raghu Angadi updated HADOOP-1138:
---------------------------------

    Attachment: HADOOP-1138.patch


new config {{dfs.report.datanode.timeout.hours}} is added.

A dead datanode is listed on webui or 'dfsadmin -report' if
 - the node is present in "dfs.hosts" file or
 - It is not listed in "dfs.hosts.exlude" and it has been inactive for less than "dfs.report.datanode.timeout.hours"

> Datanodes that are dead for a long long time should not show up in the UI
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-1138
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1138
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: Raghu Angadi
>             Fix For: 0.15.0
>
>         Attachments: HADOOP-1138.patch
>
>
> Proposal 1:
> If a include files is used, then show all nodes (dead/alive) that are listed in the includes file. If there isn't an include file, then display only nodes that have pinged this instance of the namenode.
> Proposal2:
> A config variable specifies the time duration. The namenode, on a restart, purges all datanodes that have not pinged for that time duration. The default value of this config variable can be 1 week. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-1138) Datanodes that are dead for a long long time should not show up in the UI

Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12524814 ] 

dhruba borthakur commented on HADOOP-1138:
------------------------------------------

The code looks good. A few comments:

1. FSnamesystem.getDatanodeListForReport excludes nodes that are listed in dfs.hosts.exclude. Maybe a better option wold be to show them with a status of "Excluded". Currently, it shows "Decommisioned" or "In Service".

2. The comment in FSnamesystem.getDatanodeListForReport talks about "dfs.report.datanode.timeout.day" but it should be "dfs.report.datanode.timeout.hours".

3. Maybe a unit test case that tests this functionality would be really nice. 

> Datanodes that are dead for a long long time should not show up in the UI
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-1138
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1138
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: Raghu Angadi
>             Fix For: 0.15.0
>
>         Attachments: HADOOP-1138.patch
>
>
> Proposal 1:
> If a include files is used, then show all nodes (dead/alive) that are listed in the includes file. If there isn't an include file, then display only nodes that have pinged this instance of the namenode.
> Proposal2:
> A config variable specifies the time duration. The namenode, on a restart, purges all datanodes that have not pinged for that time duration. The default value of this config variable can be 1 week. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-1138) Datanodes that are dead for a long long time should not show up in the UI

Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12519159 ] 

dhruba borthakur commented on HADOOP-1138:
------------------------------------------

It is better to remove DatanodeDescriptors from the fsimage and store only that much information that is required to generate a new storageid for new datanodes.

> Datanodes that are dead for a long long time should not show up in the UI
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-1138
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1138
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: Raghu Angadi
>             Fix For: 0.15.0
>
>
> Proposal 1:
> If a include files is used, then show all nodes (dead/alive) that are listed in the includes file. If there isn't an include file, then display only nodes that have pinged this instance of the namenode.
> Proposal2:
> A config variable specifies the time duration. The namenode, on a restart, purges all datanodes that have not pinged for that time duration. The default value of this config variable can be 1 week. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-1138) Datanodes that are dead for a long long time should not show up in the UI

Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12524828 ] 

dhruba borthakur commented on HADOOP-1138:
------------------------------------------

+1. 

> Datanodes that are dead for a long long time should not show up in the UI
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-1138
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1138
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: Raghu Angadi
>             Fix For: 0.15.0
>
>         Attachments: HADOOP-1138.patch, HADOOP-1138.patch
>
>
> Proposal 1:
> If a include files is used, then show all nodes (dead/alive) that are listed in the includes file. If there isn't an include file, then display only nodes that have pinged this instance of the namenode.
> Proposal2:
> A config variable specifies the time duration. The namenode, on a restart, purges all datanodes that have not pinged for that time duration. The default value of this config variable can be 1 week. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-1138) Datanodes that are dead for a long long time should not show up in the UI

Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12525247 ] 

Hadoop QA commented on HADOOP-1138:
-----------------------------------

+1

http://issues.apache.org/jira/secure/attachment/12365214/HADOOP-1138.patch applied and successfully tested against trunk revision r573081.

Test results:   http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/693/testReport/
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/693/console

> Datanodes that are dead for a long long time should not show up in the UI
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-1138
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1138
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: Raghu Angadi
>             Fix For: 0.15.0
>
>         Attachments: HADOOP-1138.patch, HADOOP-1138.patch, HADOOP-1138.patch
>
>
> Proposal 1:
> If a include files is used, then show all nodes (dead/alive) that are listed in the includes file. If there isn't an include file, then display only nodes that have pinged this instance of the namenode.
> Proposal2:
> A config variable specifies the time duration. The namenode, on a restart, purges all datanodes that have not pinged for that time duration. The default value of this config variable can be 1 week. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-1138) Datanodes that are dead for a long long time should not show up in the UI

Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Raghu Angadi updated HADOOP-1138:
---------------------------------

    Status: Patch Available  (was: Open)

Thanks Dhruba.

> Datanodes that are dead for a long long time should not show up in the UI
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-1138
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1138
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: Raghu Angadi
>             Fix For: 0.15.0
>
>         Attachments: HADOOP-1138.patch, HADOOP-1138.patch
>
>
> Proposal 1:
> If a include files is used, then show all nodes (dead/alive) that are listed in the includes file. If there isn't an include file, then display only nodes that have pinged this instance of the namenode.
> Proposal2:
> A config variable specifies the time duration. The namenode, on a restart, purges all datanodes that have not pinged for that time duration. The default value of this config variable can be 1 week. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-1138) Datanodes that are dead for a long long time should not show up in the UI

Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12519182 ] 

Raghu Angadi commented on HADOOP-1138:
--------------------------------------


Currently NameNode does not need storage ids across restarts (we think. let me know otherwise). Based on this, proposal is to not to store *any* datanode information in the fsimage. When namenode restarts, it just generates a new storage id for each datanode that registers. Rest of the storageid/datanode behavior remains the same as now.

For webUI, proposals in the description are fine.


> Datanodes that are dead for a long long time should not show up in the UI
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-1138
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1138
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: Raghu Angadi
>             Fix For: 0.15.0
>
>
> Proposal 1:
> If a include files is used, then show all nodes (dead/alive) that are listed in the includes file. If there isn't an include file, then display only nodes that have pinged this instance of the namenode.
> Proposal2:
> A config variable specifies the time duration. The namenode, on a restart, purges all datanodes that have not pinged for that time duration. The default value of this config variable can be 1 week. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-1138) Datanodes that are dead for a long long time should not show up in the UI

Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Raghu Angadi updated HADOOP-1138:
---------------------------------

    Resolution: Won't Fix
        Status: Resolved  (was: Patch Available)

This is not considered necessary after HADOO-1762.

> Datanodes that are dead for a long long time should not show up in the UI
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-1138
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1138
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: Raghu Angadi
>             Fix For: 0.15.0
>
>         Attachments: HADOOP-1138.patch, HADOOP-1138.patch, HADOOP-1138.patch
>
>
> Proposal 1:
> If a include files is used, then show all nodes (dead/alive) that are listed in the includes file. If there isn't an include file, then display only nodes that have pinged this instance of the namenode.
> Proposal2:
> A config variable specifies the time duration. The namenode, on a restart, purges all datanodes that have not pinged for that time duration. The default value of this config variable can be 1 week. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-1138) Datanodes that are dead for a long long time should not show up in the UI

Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12524948 ] 

Hadoop QA commented on HADOOP-1138:
-----------------------------------

-1, build or testing failed

2 attempts failed to build and test the latest attachment http://issues.apache.org/jira/secure/attachment/12365101/HADOOP-1138.patch against trunk revision r572826.

Test results:   http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/682/testReport/
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/682/console

Please note that this message is automatically generated and may represent a problem with the automation system and not the patch.

> Datanodes that are dead for a long long time should not show up in the UI
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-1138
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1138
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: Raghu Angadi
>             Fix For: 0.15.0
>
>         Attachments: HADOOP-1138.patch, HADOOP-1138.patch
>
>
> Proposal 1:
> If a include files is used, then show all nodes (dead/alive) that are listed in the includes file. If there isn't an include file, then display only nodes that have pinged this instance of the namenode.
> Proposal2:
> A config variable specifies the time duration. The namenode, on a restart, purges all datanodes that have not pinged for that time duration. The default value of this config variable can be 1 week. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (HADOOP-1138) Datanodes that are dead for a long long time should not show up in the UI

Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

dhruba borthakur updated HADOOP-1138:
-------------------------------------

    Fix Version/s: 0.15.0
         Assignee: Raghu Angadi  (was: dhruba borthakur)

> Datanodes that are dead for a long long time should not show up in the UI
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-1138
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1138
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: Raghu Angadi
>             Fix For: 0.15.0
>
>
> Proposal 1:
> If a include files is used, then show all nodes (dead/alive) that are listed in the includes file. If there isn't an include file, then display only nodes that have pinged this instance of the namenode.
> Proposal2:
> A config variable specifies the time duration. The namenode, on a restart, purges all datanodes that have not pinged for that time duration. The default value of this config variable can be 1 week. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (HADOOP-1138) Datanodes that are dead for a long long time should not show up in the UI

Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HADOOP-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12529288 ] 

Raghu Angadi commented on HADOOP-1138:
--------------------------------------


There is too much stuff in this jira. I filed a new one HADOOP-1933. Should nodes from both the lists be listed or only from include list... please comment in HADOOP-1933.


> Datanodes that are dead for a long long time should not show up in the UI
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-1138
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1138
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: Raghu Angadi
>             Fix For: 0.15.0
>
>         Attachments: HADOOP-1138.patch, HADOOP-1138.patch, HADOOP-1138.patch
>
>
> Proposal 1:
> If a include files is used, then show all nodes (dead/alive) that are listed in the includes file. If there isn't an include file, then display only nodes that have pinged this instance of the namenode.
> Proposal2:
> A config variable specifies the time duration. The namenode, on a restart, purges all datanodes that have not pinged for that time duration. The default value of this config variable can be 1 week. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Reopened: (HADOOP-1138) Datanodes that are dead for a long long time should not show up in the UI

Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HADOOP-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

dhruba borthakur reopened HADOOP-1138:
--------------------------------------


I think that if the includes list or the excludes list is present, these nodes should always show up in the UI. This allows the administrator to accurately know the state of his cluster.(  However, we do not need to automatically remove datanodes from the UI if they are dead for a certain period).

> Datanodes that are dead for a long long time should not show up in the UI
> -------------------------------------------------------------------------
>
>                 Key: HADOOP-1138
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1138
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: Raghu Angadi
>             Fix For: 0.15.0
>
>         Attachments: HADOOP-1138.patch, HADOOP-1138.patch, HADOOP-1138.patch
>
>
> Proposal 1:
> If a include files is used, then show all nodes (dead/alive) that are listed in the includes file. If there isn't an include file, then display only nodes that have pinged this instance of the namenode.
> Proposal2:
> A config variable specifies the time duration. The namenode, on a restart, purges all datanodes that have not pinged for that time duration. The default value of this config variable can be 1 week. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.