You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Nicolas Spiegelberg (JIRA)" <ji...@apache.org> on 2010/07/22 02:51:52 UTC
[jira] Created: (HBASE-2862) Name DFSClient for Improved Debugging
Name DFSClient for Improved Debugging
-------------------------------------
Key: HBASE-2862
URL: https://issues.apache.org/jira/browse/HBASE-2862
Project: HBase
Issue Type: Improvement
Components: master, regionserver
Reporter: Nicolas Spiegelberg
Assignee: Nicolas Spiegelberg
Priority: Trivial
Fix For: 0.90.0
Reported by Hairong. We had an HDFS error on our test cluster. It was hard to debug the HDFS NameNode logs because there was no way to map DFClient => RegionServer. Hadoop guys added a hacky config value, "mapred.task.id", which allows you to add a suffix to the DFSClient ID for logging. We should piggyback upon that for our HLog/HFile code
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2862) Name DFSClient for Improved
Debugging
Posted by "Nicolas Spiegelberg (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12891224#action_12891224 ]
Nicolas Spiegelberg commented on HBASE-2862:
--------------------------------------------
I'd prefer some name prefix, whether it's HRS/RegionServer/HRegionServer/*whatever*. It's nice to have a clearly-unique name per process for situations where we have multiple HDFS Clients per machine. This is compounded by looking at HDFS logs during unit tests or psuedo-distributed testing.
> Name DFSClient for Improved Debugging
> -------------------------------------
>
> Key: HBASE-2862
> URL: https://issues.apache.org/jira/browse/HBASE-2862
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Nicolas Spiegelberg
> Assignee: Nicolas Spiegelberg
> Priority: Trivial
> Fix For: 0.90.0
>
> Attachments: HBASE-2862.patch
>
>
> Reported by Hairong. We had an HDFS error on our test cluster. It was hard to debug the HDFS NameNode logs because there was no way to map DFClient => RegionServer. Hadoop guys added a hacky config value, "mapred.task.id", which allows you to add a suffix to the DFSClient ID for logging. We should piggyback upon that for our HLog/HFile code
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2862) Name DFSClient for Improved
Debugging
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12890999#action_12890999 ]
stack commented on HBASE-2862:
------------------------------
This sounds great. You know what issue it is N that has the patch we can rob from?
> Name DFSClient for Improved Debugging
> -------------------------------------
>
> Key: HBASE-2862
> URL: https://issues.apache.org/jira/browse/HBASE-2862
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Nicolas Spiegelberg
> Assignee: Nicolas Spiegelberg
> Priority: Trivial
> Fix For: 0.90.0
>
>
> Reported by Hairong. We had an HDFS error on our test cluster. It was hard to debug the HDFS NameNode logs because there was no way to map DFClient => RegionServer. Hadoop guys added a hacky config value, "mapred.task.id", which allows you to add a suffix to the DFSClient ID for logging. We should piggyback upon that for our HLog/HFile code
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2862) Name DFSClient for Improved
Debugging
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12891195#action_12891195 ]
stack commented on HBASE-2862:
------------------------------
I like this patch.
May I suggest the following N:
In HRS edit you do this,
{code}
+ this.conf.set("mapred.task.id",
+ "HRegionServer_" + this.serverInfo.getServerAddress().toString());
{code}
On commit, mind if I change the String to instead be HServerAddress.getServerName? We're trying to standardize on what we call the HRS. getServerAddress is kinda ugly -- its <hostname> ',' <port> ',' <startcode> -- but it has all we'd ever need in it and its being used in our logs, both local to RS and its how the HRS shows in the master log. I see an advantage to having the HRS name shown in NN logs be same as is shown in ours (getServerAddress also adds startcode so we'll be able to distingush file actions either side of a crash).
Master name would remain as you have it -- we don't have an equivalent getServerAddress for master (though maybe we should since there can be different masters over time).
If amenable to the above, no need to make a new patch. I'll just make change on commit.
Thanks.
> Name DFSClient for Improved Debugging
> -------------------------------------
>
> Key: HBASE-2862
> URL: https://issues.apache.org/jira/browse/HBASE-2862
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Nicolas Spiegelberg
> Assignee: Nicolas Spiegelberg
> Priority: Trivial
> Fix For: 0.90.0
>
> Attachments: HBASE-2862.patch
>
>
> Reported by Hairong. We had an HDFS error on our test cluster. It was hard to debug the HDFS NameNode logs because there was no way to map DFClient => RegionServer. Hadoop guys added a hacky config value, "mapred.task.id", which allows you to add a suffix to the DFSClient ID for logging. We should piggyback upon that for our HLog/HFile code
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-2862) Name DFSClient for Improved Debugging
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack resolved HBASE-2862.
--------------------------
Hadoop Flags: [Reviewed]
Resolution: Fixed
Thanks for nice patch Nicolas.
> Name DFSClient for Improved Debugging
> -------------------------------------
>
> Key: HBASE-2862
> URL: https://issues.apache.org/jira/browse/HBASE-2862
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Nicolas Spiegelberg
> Assignee: Nicolas Spiegelberg
> Priority: Trivial
> Fix For: 0.90.0
>
> Attachments: HBASE-2862.patch
>
>
> Reported by Hairong. We had an HDFS error on our test cluster. It was hard to debug the HDFS NameNode logs because there was no way to map DFClient => RegionServer. Hadoop guys added a hacky config value, "mapred.task.id", which allows you to add a suffix to the DFSClient ID for logging. We should piggyback upon that for our HLog/HFile code
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-2862) Name DFSClient for Improved Debugging
Posted by "Nicolas Spiegelberg (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nicolas Spiegelberg updated HBASE-2862:
---------------------------------------
Attachment: HBASE-2862.patch
This is the only patch needed. Config option is (at the least) supported in 0.20+
> Name DFSClient for Improved Debugging
> -------------------------------------
>
> Key: HBASE-2862
> URL: https://issues.apache.org/jira/browse/HBASE-2862
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
> Reporter: Nicolas Spiegelberg
> Assignee: Nicolas Spiegelberg
> Priority: Trivial
> Fix For: 0.90.0
>
> Attachments: HBASE-2862.patch
>
>
> Reported by Hairong. We had an HDFS error on our test cluster. It was hard to debug the HDFS NameNode logs because there was no way to map DFClient => RegionServer. Hadoop guys added a hacky config value, "mapred.task.id", which allows you to add a suffix to the DFSClient ID for logging. We should piggyback upon that for our HLog/HFile code
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.