You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (Created) (JIRA)" <ji...@apache.org> on 2012/03/30 18:20:31 UTC
[jira] [Created] (HADOOP-8231) Make topologies easier to set up and
debug
Make topologies easier to set up and debug
------------------------------------------
Key: HADOOP-8231
URL: https://issues.apache.org/jira/browse/HADOOP-8231
Project: Hadoop Common
Issue Type: Improvement
Components: util
Affects Versions: 0.23.1
Reporter: Steve Loughran
Priority: Minor
Topology scripts are a source of problems as they
# are site-specific.
# hard to get right.
# can have adverse consequences on cluster operation when they go wrong.
This issue is to group up the features needed to make it easier for ops people to get their scripts up and running.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8231) Make topologies easier to set up and
debug
Posted by "Steve Loughran (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Loughran updated HADOOP-8231:
-----------------------------------
Status: Open (was: Patch Available)
patch was to go into sub task
> Make topologies easier to set up and debug
> ------------------------------------------
>
> Key: HADOOP-8231
> URL: https://issues.apache.org/jira/browse/HADOOP-8231
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
> Affects Versions: 0.23.1
> Reporter: Steve Loughran
> Priority: Minor
> Fix For: 3.0.0
>
>
> Topology scripts are a source of problems as they
> # are site-specific.
> # hard to get right.
> # can have adverse consequences on cluster operation when they go wrong.
> This issue is to group up the features needed to make it easier for ops people to get their scripts up and running.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8231) Make topologies easier to set up and
debug
Posted by "Steve Loughran (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Loughran updated HADOOP-8231:
-----------------------------------
Attachment: HADOOP-8232.patch
This is the implementation class with some changes to make the output more readable.
# It does not include any tests; the process for testing entry points is new to me.
# it does not patch the bin/hadoop script
> Make topologies easier to set up and debug
> ------------------------------------------
>
> Key: HADOOP-8231
> URL: https://issues.apache.org/jira/browse/HADOOP-8231
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
> Affects Versions: 0.23.1
> Reporter: Steve Loughran
> Priority: Minor
> Attachments: HADOOP-8232.patch
>
>
> Topology scripts are a source of problems as they
> # are site-specific.
> # hard to get right.
> # can have adverse consequences on cluster operation when they go wrong.
> This issue is to group up the features needed to make it easier for ops people to get their scripts up and running.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8231) Make topologies easier to set up
and debug
Posted by "Steve Loughran (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242583#comment-13242583 ]
Steve Loughran commented on HADOOP-8231:
----------------------------------------
Another way to simplify the process would be add JSP pages to the NN and RM, so that the topologies can be listed. The output could be an HTML table or piece of JSON. Clean HTML is easier to generate with JSP than json; JSP easier to test by parsing the output.
The simplest is just to output text/plain and print the topology information created by {{AbstractDNSToSwitchMapping.dumpTopology()}}, which is what is printed out at the end of every successful run of the HADOOP-8232 resolver runs.
> Make topologies easier to set up and debug
> ------------------------------------------
>
> Key: HADOOP-8231
> URL: https://issues.apache.org/jira/browse/HADOOP-8231
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
> Affects Versions: 0.23.1
> Reporter: Steve Loughran
> Priority: Minor
> Fix For: 3.0.0
>
>
> Topology scripts are a source of problems as they
> # are site-specific.
> # hard to get right.
> # can have adverse consequences on cluster operation when they go wrong.
> This issue is to group up the features needed to make it easier for ops people to get their scripts up and running.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8231) Make topologies easier to set up and
debug
Posted by "Steve Loughran (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Loughran updated HADOOP-8231:
-----------------------------------
Fix Version/s: 3.0.0
Status: Patch Available (was: Open)
not ready for committing yet, sending to jenkins
> Make topologies easier to set up and debug
> ------------------------------------------
>
> Key: HADOOP-8231
> URL: https://issues.apache.org/jira/browse/HADOOP-8231
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
> Affects Versions: 0.23.1
> Reporter: Steve Loughran
> Priority: Minor
> Fix For: 3.0.0
>
> Attachments: HADOOP-8232.patch
>
>
> Topology scripts are a source of problems as they
> # are site-specific.
> # hard to get right.
> # can have adverse consequences on cluster operation when they go wrong.
> This issue is to group up the features needed to make it easier for ops people to get their scripts up and running.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8231) Make topologies easier to set up and
debug
Posted by "Steve Loughran (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Loughran updated HADOOP-8231:
-----------------------------------
Attachment: (was: HADOOP-8232.patch)
> Make topologies easier to set up and debug
> ------------------------------------------
>
> Key: HADOOP-8231
> URL: https://issues.apache.org/jira/browse/HADOOP-8231
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
> Affects Versions: 0.23.1
> Reporter: Steve Loughran
> Priority: Minor
> Fix For: 3.0.0
>
>
> Topology scripts are a source of problems as they
> # are site-specific.
> # hard to get right.
> # can have adverse consequences on cluster operation when they go wrong.
> This issue is to group up the features needed to make it easier for ops people to get their scripts up and running.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8231) Make topologies easier to set up
and debug
Posted by "liang xie (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502996#comment-13502996 ]
liang xie commented on HADOOP-8231:
-----------------------------------
"hdfs dfsadmin -printTopology" is a good way to dump topology state
> Make topologies easier to set up and debug
> ------------------------------------------
>
> Key: HADOOP-8231
> URL: https://issues.apache.org/jira/browse/HADOOP-8231
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
> Affects Versions: 0.23.1
> Reporter: Steve Loughran
> Priority: Minor
> Fix For: 3.0.0
>
>
> Topology scripts are a source of problems as they
> # are site-specific.
> # hard to get right.
> # can have adverse consequences on cluster operation when they go wrong.
> This issue is to group up the features needed to make it easier for ops people to get their scripts up and running.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira