You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org> on 2009/10/19 20:55:59 UTC
[jira] Created: (HBASE-1918) Don't do DNS resolving in .META.
scanner for each row
Don't do DNS resolving in .META. scanner for each row
-----------------------------------------------------
Key: HBASE-1918
URL: https://issues.apache.org/jira/browse/HBASE-1918
Project: Hadoop HBase
Issue Type: Improvement
Affects Versions: 0.20.1, 0.19.3
Reporter: Jean-Daniel Cryans
Assignee: Jean-Daniel Cryans
Fix For: 0.20.2, 0.21.0
While debugging a slow HBase on my new Mac, I found that my issue was that DNS resolving was taking 10 seconds. Since Java 1.5, many people complained about that and there's even a very commented issue (http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5092063) that's closed but still "active". In fact the real issue here is that when you a bad DNS resolving (my router was acting on me, it's dead now) it waits for 10 seconds or something like that (see previous link). So I was running PE and my meta scanner was being slower and slower. In fact, everything was slow, even starting the shell. So using jps I found that we are resolving DNS for each scanned row in META in checkAssigned here:
{code}
if (sa != null && sa.length() > 0) {
serverName = HServerInfo.getServerName(sa, sc);
}
HServerInfo storedInfo = null;
{code}
HSI.getServerName here creates a new HServerAddress which resolves DNS in its depths and it's not even necessary, we use it just to split the server IP and port.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1918) Don't do DNS resolving in .META.
scanner for each row
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12767675#action_12767675 ]
Andrew Purtell commented on HBASE-1918:
---------------------------------------
The -v3 patch fixes the issues I'm seeing on 0.20 branch also.
> Don't do DNS resolving in .META. scanner for each row
> -----------------------------------------------------
>
> Key: HBASE-1918
> URL: https://issues.apache.org/jira/browse/HBASE-1918
> Project: Hadoop HBase
> Issue Type: Improvement
> Affects Versions: 0.19.3, 0.20.1
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.20.2, 0.21.0
>
> Attachments: HBASE-1918-2.patch, HBASE-1918-3.patch, HBASE-1918.patch
>
>
> While debugging a slow HBase on my new Mac, I found that my issue was that DNS resolving was taking 10 seconds. Since Java 1.5, many people complained about that and there's even a very commented issue (http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5092063) that's closed but still "active". In fact the real issue here is that when you a bad DNS resolving (my router was acting on me, it's dead now) it waits for 10 seconds or something like that (see previous link). So I was running PE and my meta scanner was being slower and slower. In fact, everything was slow, even starting the shell. So using jps I found that we are resolving DNS for each scanned row in META in checkAssigned here:
> {code}
> if (sa != null && sa.length() > 0) {
> serverName = HServerInfo.getServerName(sa, sc);
> }
> HServerInfo storedInfo = null;
> {code}
> HSI.getServerName here creates a new HServerAddress which resolves DNS in its depths and it's not even necessary, we use it just to split the server IP and port.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1918) Don't do DNS resolving in .META.
scanner for each row
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12767459#action_12767459 ]
Jonathan Gray commented on HBASE-1918:
--------------------------------------
Patch looks good, JD. +1 for commit to branch and trunk
> Don't do DNS resolving in .META. scanner for each row
> -----------------------------------------------------
>
> Key: HBASE-1918
> URL: https://issues.apache.org/jira/browse/HBASE-1918
> Project: Hadoop HBase
> Issue Type: Improvement
> Affects Versions: 0.19.3, 0.20.1
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.20.2, 0.21.0
>
> Attachments: HBASE-1918.patch
>
>
> While debugging a slow HBase on my new Mac, I found that my issue was that DNS resolving was taking 10 seconds. Since Java 1.5, many people complained about that and there's even a very commented issue (http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5092063) that's closed but still "active". In fact the real issue here is that when you a bad DNS resolving (my router was acting on me, it's dead now) it waits for 10 seconds or something like that (see previous link). So I was running PE and my meta scanner was being slower and slower. In fact, everything was slow, even starting the shell. So using jps I found that we are resolving DNS for each scanned row in META in checkAssigned here:
> {code}
> if (sa != null && sa.length() > 0) {
> serverName = HServerInfo.getServerName(sa, sc);
> }
> HServerInfo storedInfo = null;
> {code}
> HSI.getServerName here creates a new HServerAddress which resolves DNS in its depths and it's not even necessary, we use it just to split the server IP and port.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-1918) Don't do DNS resolving in .META.
scanner for each row
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans resolved HBASE-1918.
---------------------------------------
Resolution: Fixed
Committed. Sorry for the problems, unfortunately my machine's configuration was different in a way that the tests didn't fail for me on the second patch.
> Don't do DNS resolving in .META. scanner for each row
> -----------------------------------------------------
>
> Key: HBASE-1918
> URL: https://issues.apache.org/jira/browse/HBASE-1918
> Project: Hadoop HBase
> Issue Type: Improvement
> Affects Versions: 0.19.3, 0.20.1
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.20.2, 0.21.0
>
> Attachments: HBASE-1918-2.patch, HBASE-1918-3.patch, HBASE-1918.patch
>
>
> While debugging a slow HBase on my new Mac, I found that my issue was that DNS resolving was taking 10 seconds. Since Java 1.5, many people complained about that and there's even a very commented issue (http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5092063) that's closed but still "active". In fact the real issue here is that when you a bad DNS resolving (my router was acting on me, it's dead now) it waits for 10 seconds or something like that (see previous link). So I was running PE and my meta scanner was being slower and slower. In fact, everything was slow, even starting the shell. So using jps I found that we are resolving DNS for each scanned row in META in checkAssigned here:
> {code}
> if (sa != null && sa.length() > 0) {
> serverName = HServerInfo.getServerName(sa, sc);
> }
> HServerInfo storedInfo = null;
> {code}
> HSI.getServerName here creates a new HServerAddress which resolves DNS in its depths and it's not even necessary, we use it just to split the server IP and port.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1918) Don't do DNS resolving in .META.
scanner for each row
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans updated HBASE-1918:
--------------------------------------
Attachment: HBASE-1918.patch
Patch that fixes the mentioned problem, passes tests.
> Don't do DNS resolving in .META. scanner for each row
> -----------------------------------------------------
>
> Key: HBASE-1918
> URL: https://issues.apache.org/jira/browse/HBASE-1918
> Project: Hadoop HBase
> Issue Type: Improvement
> Affects Versions: 0.19.3, 0.20.1
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.20.2, 0.21.0
>
> Attachments: HBASE-1918.patch
>
>
> While debugging a slow HBase on my new Mac, I found that my issue was that DNS resolving was taking 10 seconds. Since Java 1.5, many people complained about that and there's even a very commented issue (http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5092063) that's closed but still "active". In fact the real issue here is that when you a bad DNS resolving (my router was acting on me, it's dead now) it waits for 10 seconds or something like that (see previous link). So I was running PE and my meta scanner was being slower and slower. In fact, everything was slow, even starting the shell. So using jps I found that we are resolving DNS for each scanned row in META in checkAssigned here:
> {code}
> if (sa != null && sa.length() > 0) {
> serverName = HServerInfo.getServerName(sa, sc);
> }
> HServerInfo storedInfo = null;
> {code}
> HSI.getServerName here creates a new HServerAddress which resolves DNS in its depths and it's not even necessary, we use it just to split the server IP and port.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-1918) Don't do DNS resolving in .META.
scanner for each row
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans resolved HBASE-1918.
---------------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Committed to branch and trunk.
> Don't do DNS resolving in .META. scanner for each row
> -----------------------------------------------------
>
> Key: HBASE-1918
> URL: https://issues.apache.org/jira/browse/HBASE-1918
> Project: Hadoop HBase
> Issue Type: Improvement
> Affects Versions: 0.19.3, 0.20.1
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.20.2, 0.21.0
>
> Attachments: HBASE-1918-2.patch, HBASE-1918.patch
>
>
> While debugging a slow HBase on my new Mac, I found that my issue was that DNS resolving was taking 10 seconds. Since Java 1.5, many people complained about that and there's even a very commented issue (http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5092063) that's closed but still "active". In fact the real issue here is that when you a bad DNS resolving (my router was acting on me, it's dead now) it waits for 10 seconds or something like that (see previous link). So I was running PE and my meta scanner was being slower and slower. In fact, everything was slow, even starting the shell. So using jps I found that we are resolving DNS for each scanned row in META in checkAssigned here:
> {code}
> if (sa != null && sa.length() > 0) {
> serverName = HServerInfo.getServerName(sa, sc);
> }
> HServerInfo storedInfo = null;
> {code}
> HSI.getServerName here creates a new HServerAddress which resolves DNS in its depths and it's not even necessary, we use it just to split the server IP and port.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1918) Don't do DNS resolving in .META.
scanner for each row
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12767493#action_12767493 ]
stack commented on HBASE-1918:
------------------------------
+1 but mildly interested in what happens if no port on the serverAddress...
We need an issue for fact that on each heartbeat, a dns lookup is also done.
> Don't do DNS resolving in .META. scanner for each row
> -----------------------------------------------------
>
> Key: HBASE-1918
> URL: https://issues.apache.org/jira/browse/HBASE-1918
> Project: Hadoop HBase
> Issue Type: Improvement
> Affects Versions: 0.19.3, 0.20.1
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.20.2, 0.21.0
>
> Attachments: HBASE-1918.patch
>
>
> While debugging a slow HBase on my new Mac, I found that my issue was that DNS resolving was taking 10 seconds. Since Java 1.5, many people complained about that and there's even a very commented issue (http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5092063) that's closed but still "active". In fact the real issue here is that when you a bad DNS resolving (my router was acting on me, it's dead now) it waits for 10 seconds or something like that (see previous link). So I was running PE and my meta scanner was being slower and slower. In fact, everything was slow, even starting the shell. So using jps I found that we are resolving DNS for each scanned row in META in checkAssigned here:
> {code}
> if (sa != null && sa.length() > 0) {
> serverName = HServerInfo.getServerName(sa, sc);
> }
> HServerInfo storedInfo = null;
> {code}
> HSI.getServerName here creates a new HServerAddress which resolves DNS in its depths and it's not even necessary, we use it just to split the server IP and port.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1918) Don't do DNS resolving in .META.
scanner for each row
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans updated HBASE-1918:
--------------------------------------
Attachment: HBASE-1918-3.patch
I tried this patch on trunk and it passed on Hudson. Now we cache the DNS resolution enabling us to instantiate a lot less objects and saving us from DNS negative caching. That's ok with you guys?
> Don't do DNS resolving in .META. scanner for each row
> -----------------------------------------------------
>
> Key: HBASE-1918
> URL: https://issues.apache.org/jira/browse/HBASE-1918
> Project: Hadoop HBase
> Issue Type: Improvement
> Affects Versions: 0.19.3, 0.20.1
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.20.2, 0.21.0
>
> Attachments: HBASE-1918-2.patch, HBASE-1918-3.patch, HBASE-1918.patch
>
>
> While debugging a slow HBase on my new Mac, I found that my issue was that DNS resolving was taking 10 seconds. Since Java 1.5, many people complained about that and there's even a very commented issue (http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5092063) that's closed but still "active". In fact the real issue here is that when you a bad DNS resolving (my router was acting on me, it's dead now) it waits for 10 seconds or something like that (see previous link). So I was running PE and my meta scanner was being slower and slower. In fact, everything was slow, even starting the shell. So using jps I found that we are resolving DNS for each scanned row in META in checkAssigned here:
> {code}
> if (sa != null && sa.length() > 0) {
> serverName = HServerInfo.getServerName(sa, sc);
> }
> HServerInfo storedInfo = null;
> {code}
> HSI.getServerName here creates a new HServerAddress which resolves DNS in its depths and it's not even necessary, we use it just to split the server IP and port.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1918) Don't do DNS resolving in .META.
scanner for each row
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12767679#action_12767679 ]
stack commented on HBASE-1918:
------------------------------
So, once cached, it can never change. I suppose this is fine. Maybe one day when we grow up and we have hbase clusters of 1k hosts and we need to support machines changing IPs, we can use something like http://www.xbill.org/dnsjava/ to manage our dns cache. Meantime, patch looks good to me. +1.
> Don't do DNS resolving in .META. scanner for each row
> -----------------------------------------------------
>
> Key: HBASE-1918
> URL: https://issues.apache.org/jira/browse/HBASE-1918
> Project: Hadoop HBase
> Issue Type: Improvement
> Affects Versions: 0.19.3, 0.20.1
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.20.2, 0.21.0
>
> Attachments: HBASE-1918-2.patch, HBASE-1918-3.patch, HBASE-1918.patch
>
>
> While debugging a slow HBase on my new Mac, I found that my issue was that DNS resolving was taking 10 seconds. Since Java 1.5, many people complained about that and there's even a very commented issue (http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5092063) that's closed but still "active". In fact the real issue here is that when you a bad DNS resolving (my router was acting on me, it's dead now) it waits for 10 seconds or something like that (see previous link). So I was running PE and my meta scanner was being slower and slower. In fact, everything was slow, even starting the shell. So using jps I found that we are resolving DNS for each scanned row in META in checkAssigned here:
> {code}
> if (sa != null && sa.length() > 0) {
> serverName = HServerInfo.getServerName(sa, sc);
> }
> HServerInfo storedInfo = null;
> {code}
> HSI.getServerName here creates a new HServerAddress which resolves DNS in its depths and it's not even necessary, we use it just to split the server IP and port.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1918) Don't do DNS resolving in .META.
scanner for each row
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans updated HBASE-1918:
--------------------------------------
Attachment: HBASE-1918-2.patch
Something like this patch.
> Don't do DNS resolving in .META. scanner for each row
> -----------------------------------------------------
>
> Key: HBASE-1918
> URL: https://issues.apache.org/jira/browse/HBASE-1918
> Project: Hadoop HBase
> Issue Type: Improvement
> Affects Versions: 0.19.3, 0.20.1
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.20.2, 0.21.0
>
> Attachments: HBASE-1918-2.patch, HBASE-1918.patch
>
>
> While debugging a slow HBase on my new Mac, I found that my issue was that DNS resolving was taking 10 seconds. Since Java 1.5, many people complained about that and there's even a very commented issue (http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5092063) that's closed but still "active". In fact the real issue here is that when you a bad DNS resolving (my router was acting on me, it's dead now) it waits for 10 seconds or something like that (see previous link). So I was running PE and my meta scanner was being slower and slower. In fact, everything was slow, even starting the shell. So using jps I found that we are resolving DNS for each scanned row in META in checkAssigned here:
> {code}
> if (sa != null && sa.length() > 0) {
> serverName = HServerInfo.getServerName(sa, sc);
> }
> HServerInfo storedInfo = null;
> {code}
> HSI.getServerName here creates a new HServerAddress which resolves DNS in its depths and it's not even necessary, we use it just to split the server IP and port.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1918) Don't do DNS resolving in .META.
scanner for each row
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12767503#action_12767503 ]
Jean-Daniel Cryans commented on HBASE-1918:
-------------------------------------------
Thanks guys for looking at the patch.
Stack, here's the code in HSA that was called:
{code}
int colonIndex = hostAndPort.lastIndexOf(':');
if(colonIndex < 0) {
throw new IllegalArgumentException("Not a host:port pair: " + hostAndPort);
}
String host = hostAndPort.substring(0, colonIndex);
int port =
Integer.valueOf(hostAndPort.substring(colonIndex + 1)).intValue();
{code}
At first I tried to put it in a helper method but the code is so simple I could just copy it over so we'll have the same behavior.
> Don't do DNS resolving in .META. scanner for each row
> -----------------------------------------------------
>
> Key: HBASE-1918
> URL: https://issues.apache.org/jira/browse/HBASE-1918
> Project: Hadoop HBase
> Issue Type: Improvement
> Affects Versions: 0.19.3, 0.20.1
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.20.2, 0.21.0
>
> Attachments: HBASE-1918.patch
>
>
> While debugging a slow HBase on my new Mac, I found that my issue was that DNS resolving was taking 10 seconds. Since Java 1.5, many people complained about that and there's even a very commented issue (http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5092063) that's closed but still "active". In fact the real issue here is that when you a bad DNS resolving (my router was acting on me, it's dead now) it waits for 10 seconds or something like that (see previous link). So I was running PE and my meta scanner was being slower and slower. In fact, everything was slow, even starting the shell. So using jps I found that we are resolving DNS for each scanned row in META in checkAssigned here:
> {code}
> if (sa != null && sa.length() > 0) {
> serverName = HServerInfo.getServerName(sa, sc);
> }
> HServerInfo storedInfo = null;
> {code}
> HSI.getServerName here creates a new HServerAddress which resolves DNS in its depths and it's not even necessary, we use it just to split the server IP and port.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Reopened: (HBASE-1918) Don't do DNS resolving in .META.
scanner for each row
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans reopened HBASE-1918:
---------------------------------------
Well it seems that some resolving is needed. From some debug I added:
{code}
HBASE-1918 debug : vesta.apache.org != 67.195.138.9
{code}
> Don't do DNS resolving in .META. scanner for each row
> -----------------------------------------------------
>
> Key: HBASE-1918
> URL: https://issues.apache.org/jira/browse/HBASE-1918
> Project: Hadoop HBase
> Issue Type: Improvement
> Affects Versions: 0.19.3, 0.20.1
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.20.2, 0.21.0
>
> Attachments: HBASE-1918-2.patch, HBASE-1918.patch
>
>
> While debugging a slow HBase on my new Mac, I found that my issue was that DNS resolving was taking 10 seconds. Since Java 1.5, many people complained about that and there's even a very commented issue (http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5092063) that's closed but still "active". In fact the real issue here is that when you a bad DNS resolving (my router was acting on me, it's dead now) it waits for 10 seconds or something like that (see previous link). So I was running PE and my meta scanner was being slower and slower. In fact, everything was slow, even starting the shell. So using jps I found that we are resolving DNS for each scanned row in META in checkAssigned here:
> {code}
> if (sa != null && sa.length() > 0) {
> serverName = HServerInfo.getServerName(sa, sc);
> }
> HServerInfo storedInfo = null;
> {code}
> HSI.getServerName here creates a new HServerAddress which resolves DNS in its depths and it's not even necessary, we use it just to split the server IP and port.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1918) Don't do DNS resolving in .META.
scanner for each row
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12767534#action_12767534 ]
Jonathan Gray commented on HBASE-1918:
--------------------------------------
+1 for commit, does not change behavior and resolves issue
> Don't do DNS resolving in .META. scanner for each row
> -----------------------------------------------------
>
> Key: HBASE-1918
> URL: https://issues.apache.org/jira/browse/HBASE-1918
> Project: Hadoop HBase
> Issue Type: Improvement
> Affects Versions: 0.19.3, 0.20.1
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.20.2, 0.21.0
>
> Attachments: HBASE-1918-2.patch, HBASE-1918.patch
>
>
> While debugging a slow HBase on my new Mac, I found that my issue was that DNS resolving was taking 10 seconds. Since Java 1.5, many people complained about that and there's even a very commented issue (http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5092063) that's closed but still "active". In fact the real issue here is that when you a bad DNS resolving (my router was acting on me, it's dead now) it waits for 10 seconds or something like that (see previous link). So I was running PE and my meta scanner was being slower and slower. In fact, everything was slow, even starting the shell. So using jps I found that we are resolving DNS for each scanned row in META in checkAssigned here:
> {code}
> if (sa != null && sa.length() > 0) {
> serverName = HServerInfo.getServerName(sa, sc);
> }
> HServerInfo storedInfo = null;
> {code}
> HSI.getServerName here creates a new HServerAddress which resolves DNS in its depths and it's not even necessary, we use it just to split the server IP and port.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.