You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Liu Jia (JIRA)" <ji...@apache.org> on 2011/08/09 04:52:27 UTC
[jira] [Created] (HBASE-4181) HConnectionManager can't find cached
HRegionInterface makes client very slow
HConnectionManager can't find cached HRegionInterface makes client very slow
----------------------------------------------------------------------------
Key: HBASE-4181
URL: https://issues.apache.org/jira/browse/HBASE-4181
Project: HBase
Issue Type: Bug
Components: client
Affects Versions: 0.90.4, 0.92.0
Reporter: Liu Jia
Priority: Critical
HRegionInterface getHRegionConnection(final String hostname,
final int port, final InetSocketAddress isa, final boolean master)
throws IOException
/////////////////////////
String rsName = isa != null ? isa.toString() : Addressing
.createHostAndPortStr(hostname, port); ////here,if isa is null, the Addressing created a address like "node41:60010"
////isa.toString():new InetSocketAddress(hostname, port).toString();
////instead of Addressing.createHostAndPortStr(hostname, port);
server = this.servers.get(rsName);
if (server == null) {
// create a unique lock for this RS (if necessary)
this.connectionLock.putIfAbsent(rsName, rsName);
// get the RS lock
synchronized (this.connectionLock.get(rsName)) {
// do one more lookup in case we were stalled above
server = this.servers.get(rsName);
if (server == null) {
try {
if (clusterId.hasId()) {
conf.set(HConstants.CLUSTER_ID, clusterId.getId());
}
// Only create isa when we need to.
InetSocketAddress address = isa != null ? isa
: new InetSocketAddress(hostname, port);
// definitely a cache miss. establish an RPC for this RS
server = (HRegionInterface) HBaseRPC.waitForProxy(
serverInterfaceClass, HRegionInterface.VERSION, address,
this.conf, this.maxRPCAttempts, this.rpcTimeout,
this.rpcTimeout);
this.servers.put(address.toString(), server); ////but here address.toString() send an address like "node41/10.61.2l.171:60010"
////so this method can never get cached address and make client request very slow ////due to it's synchronized.
} catch (RemoteException e) {
LOG.warn("RemoteException connecting to RS", e);
// Throw what the RemoteException was carrying.
throw RemoteExceptionHandler.decodeRemoteException(e);
}
}
}
///////////////////////
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4181) HConnectionManager can't find cached
HRegionInterface makes client very slow
Posted by "Liu Jia (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Liu Jia updated HBASE-4181:
---------------------------
Release Note: (was: remove DNS query part from getHRegionConnection())
> HConnectionManager can't find cached HRegionInterface makes client very slow
> ----------------------------------------------------------------------------
>
> Key: HBASE-4181
> URL: https://issues.apache.org/jira/browse/HBASE-4181
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.92.0
> Reporter: Liu Jia
> Priority: Critical
> Labels: HConnectionManager
> Fix For: 0.90.4
>
> Attachments: HBASE-4181-trunk-v2.patch, HBASE-4181-trunk-v3.patch, HBASE-4181.patch, HConnectionManager.patch
>
>
> HRegionInterface getHRegionConnection(final String hostname,
> final int port, final InetSocketAddress isa, final boolean master)
> throws IOException
> /////////////////////////
> String rsName = isa != null ? isa.toString() : Addressing
> .createHostAndPortStr(hostname, port);
>
> ////here,if isa is null, the Addressing created a address like "node41:60010"
> ////should use "isa.toString():new InetSocketAddress(hostname, port).toString();"
> ////instead of "Addressing.createHostAndPortStr(hostname, port);"
> server = this.servers.get(rsName);
> if (server == null) {
> // create a unique lock for this RS (if necessary)
> this.connectionLock.putIfAbsent(rsName, rsName);
> // get the RS lock
> synchronized (this.connectionLock.get(rsName)) {
> // do one more lookup in case we were stalled above
> server = this.servers.get(rsName);
> if (server == null) {
> try {
> if (clusterId.hasId()) {
> conf.set(HConstants.CLUSTER_ID, clusterId.getId());
> }
> // Only create isa when we need to.
> InetSocketAddress address = isa != null ? isa
> : new InetSocketAddress(hostname, port);
> // definitely a cache miss. establish an RPC for this RS
> server = (HRegionInterface) HBaseRPC.waitForProxy(
> serverInterfaceClass, HRegionInterface.VERSION, address,
> this.conf, this.maxRPCAttempts, this.rpcTimeout,
> this.rpcTimeout);
> this.servers.put(address.toString(), server);
>
> ////but here address.toString() send an address like "node41/10.61.2l.171:60010
> ////so this method can never get cached address and make client request very slow due to it's synchronized.
>
>
> } catch (RemoteException e) {
> LOG.warn("RemoteException connecting to RS", e);
> // Throw what the RemoteException was carrying.
> throw RemoteExceptionHandler.decodeRemoteException(e);
> }
> }
> }
> ///////////////////////
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4181) HConnectionManager can't find
cached HRegionInterface makes client very slow
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081544#comment-13081544 ]
jiraposter@reviews.apache.org commented on HBASE-4181:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1402/
-----------------------------------------------------------
(Updated 2011-08-09 09:41:48.573628)
Review request for hbase.
Changes
-------
Corrected formatting.
Summary
-------
HConnectionManager can't find cached HRegionInterface makes client very slow
Addressing.createHostAndPortStr(hostname, port); //the Addressing created a address like "node41:60010"
......
this.servers.put(address.toString(), server);
//but here address.toString() send an address like "node41/10.61.2l.171:60010
This addresses bug HBase-4181.
https://issues.apache.org/jira/browse/HBase-4181
Diffs (updated)
-----
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 1155226
Diff: https://reviews.apache.org/r/1402/diff
Testing
-------
Tests passed locally.
Thanks,
Jia
> HConnectionManager can't find cached HRegionInterface makes client very slow
> ----------------------------------------------------------------------------
>
> Key: HBASE-4181
> URL: https://issues.apache.org/jira/browse/HBASE-4181
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.92.0
> Reporter: Liu Jia
> Priority: Critical
> Labels: HConnectionManager
> Attachments: HBASE-4181-trunk-v2.patch, HBASE-4181.patch, HConnectionManager.patch
>
>
> HRegionInterface getHRegionConnection(final String hostname,
> final int port, final InetSocketAddress isa, final boolean master)
> throws IOException
> /////////////////////////
> String rsName = isa != null ? isa.toString() : Addressing
> .createHostAndPortStr(hostname, port);
>
> ////here,if isa is null, the Addressing created a address like "node41:60010"
> ////should use "isa.toString():new InetSocketAddress(hostname, port).toString();"
> ////instead of "Addressing.createHostAndPortStr(hostname, port);"
> server = this.servers.get(rsName);
> if (server == null) {
> // create a unique lock for this RS (if necessary)
> this.connectionLock.putIfAbsent(rsName, rsName);
> // get the RS lock
> synchronized (this.connectionLock.get(rsName)) {
> // do one more lookup in case we were stalled above
> server = this.servers.get(rsName);
> if (server == null) {
> try {
> if (clusterId.hasId()) {
> conf.set(HConstants.CLUSTER_ID, clusterId.getId());
> }
> // Only create isa when we need to.
> InetSocketAddress address = isa != null ? isa
> : new InetSocketAddress(hostname, port);
> // definitely a cache miss. establish an RPC for this RS
> server = (HRegionInterface) HBaseRPC.waitForProxy(
> serverInterfaceClass, HRegionInterface.VERSION, address,
> this.conf, this.maxRPCAttempts, this.rpcTimeout,
> this.rpcTimeout);
> this.servers.put(address.toString(), server);
>
> ////but here address.toString() send an address like "node41/10.61.2l.171:60010
> ////so this method can never get cached address and make client request very slow due to it's synchronized.
>
>
> } catch (RemoteException e) {
> LOG.warn("RemoteException connecting to RS", e);
> // Throw what the RemoteException was carrying.
> throw RemoteExceptionHandler.decodeRemoteException(e);
> }
> }
> }
> ///////////////////////
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4181) HConnectionManager can't find
cached HRegionInterface makes client very slow
Posted by "Liu Jia (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081471#comment-13081471 ]
Liu Jia commented on HBASE-4181:
--------------------------------
@Ted
With this line
{noformat}
String rsName = isa != null ? isa.toString() : Addressing
.createHostAndPortStr(hostname, port);
{noformat}
if the isa is not null, rsName will be initiated by isa.toString() method,
So the above part and the following part should be changed to InetSocketAddress.createUnresolved(host,port) together to avoid DNS query.
{noformat}
this.servers.put(address.toString(), server);
{noformat}
> HConnectionManager can't find cached HRegionInterface makes client very slow
> ----------------------------------------------------------------------------
>
> Key: HBASE-4181
> URL: https://issues.apache.org/jira/browse/HBASE-4181
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.92.0
> Reporter: Liu Jia
> Priority: Critical
> Labels: HConnectionManager
> Attachments: HBASE-4181.patch, HConnectionManager.patch
>
>
> HRegionInterface getHRegionConnection(final String hostname,
> final int port, final InetSocketAddress isa, final boolean master)
> throws IOException
> /////////////////////////
> String rsName = isa != null ? isa.toString() : Addressing
> .createHostAndPortStr(hostname, port);
>
> ////here,if isa is null, the Addressing created a address like "node41:60010"
> ////should use "isa.toString():new InetSocketAddress(hostname, port).toString();"
> ////instead of "Addressing.createHostAndPortStr(hostname, port);"
> server = this.servers.get(rsName);
> if (server == null) {
> // create a unique lock for this RS (if necessary)
> this.connectionLock.putIfAbsent(rsName, rsName);
> // get the RS lock
> synchronized (this.connectionLock.get(rsName)) {
> // do one more lookup in case we were stalled above
> server = this.servers.get(rsName);
> if (server == null) {
> try {
> if (clusterId.hasId()) {
> conf.set(HConstants.CLUSTER_ID, clusterId.getId());
> }
> // Only create isa when we need to.
> InetSocketAddress address = isa != null ? isa
> : new InetSocketAddress(hostname, port);
> // definitely a cache miss. establish an RPC for this RS
> server = (HRegionInterface) HBaseRPC.waitForProxy(
> serverInterfaceClass, HRegionInterface.VERSION, address,
> this.conf, this.maxRPCAttempts, this.rpcTimeout,
> this.rpcTimeout);
> this.servers.put(address.toString(), server);
>
> ////but here address.toString() send an address like "node41/10.61.2l.171:60010
> ////so this method can never get cached address and make client request very slow due to it's synchronized.
>
>
> } catch (RemoteException e) {
> LOG.warn("RemoteException connecting to RS", e);
> // Throw what the RemoteException was carrying.
> throw RemoteExceptionHandler.decodeRemoteException(e);
> }
> }
> }
> ///////////////////////
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4181) HConnectionManager can't find
cached HRegionInterface makes client very slow
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081844#comment-13081844 ]
Hudson commented on HBASE-4181:
-------------------------------
Integrated in HBase-TRUNK #2100 (See [https://builds.apache.org/job/HBase-TRUNK/2100/])
HBASE-4181 HConnectionManager can't find cached HRegionInterface and makes clients work very slow
stack :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
* /hbase/trunk/CHANGES.txt
> HConnectionManager can't find cached HRegionInterface makes client very slow
> ----------------------------------------------------------------------------
>
> Key: HBASE-4181
> URL: https://issues.apache.org/jira/browse/HBASE-4181
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.92.0
> Reporter: Liu Jia
> Assignee: Liu Jia
> Priority: Critical
> Labels: HConnectionManager
> Fix For: 0.92.0
>
> Attachments: HBASE-4181-trunk-v2.patch, HBASE-4181-trunk-v3.patch, HBASE-4181.patch, HConnectionManager.patch
>
>
> HRegionInterface getHRegionConnection(final String hostname,
> final int port, final InetSocketAddress isa, final boolean master)
> throws IOException
> /////////////////////////
> String rsName = isa != null ? isa.toString() : Addressing
> .createHostAndPortStr(hostname, port);
>
> ////here,if isa is null, the Addressing created a address like "node41:60010"
> ////should use "isa.toString():new InetSocketAddress(hostname, port).toString();"
> ////instead of "Addressing.createHostAndPortStr(hostname, port);"
> server = this.servers.get(rsName);
> if (server == null) {
> // create a unique lock for this RS (if necessary)
> this.connectionLock.putIfAbsent(rsName, rsName);
> // get the RS lock
> synchronized (this.connectionLock.get(rsName)) {
> // do one more lookup in case we were stalled above
> server = this.servers.get(rsName);
> if (server == null) {
> try {
> if (clusterId.hasId()) {
> conf.set(HConstants.CLUSTER_ID, clusterId.getId());
> }
> // Only create isa when we need to.
> InetSocketAddress address = isa != null ? isa
> : new InetSocketAddress(hostname, port);
> // definitely a cache miss. establish an RPC for this RS
> server = (HRegionInterface) HBaseRPC.waitForProxy(
> serverInterfaceClass, HRegionInterface.VERSION, address,
> this.conf, this.maxRPCAttempts, this.rpcTimeout,
> this.rpcTimeout);
> this.servers.put(address.toString(), server);
>
> ////but here address.toString() send an address like "node41/10.61.2l.171:60010
> ////so this method can never get cached address and make client request very slow due to it's synchronized.
>
>
> } catch (RemoteException e) {
> LOG.warn("RemoteException connecting to RS", e);
> // Throw what the RemoteException was carrying.
> throw RemoteExceptionHandler.decodeRemoteException(e);
> }
> }
> }
> ///////////////////////
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4181) HConnectionManager can't find cached
HRegionInterface makes client very slow
Posted by "Liu Jia (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Liu Jia updated HBASE-4181:
---------------------------
Attachment: HBASE-4181.patch
some modification
> HConnectionManager can't find cached HRegionInterface makes client very slow
> ----------------------------------------------------------------------------
>
> Key: HBASE-4181
> URL: https://issues.apache.org/jira/browse/HBASE-4181
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.92.0
> Reporter: Liu Jia
> Priority: Critical
> Labels: HConnectionManager
> Attachments: HBASE-4181.patch, HConnectionManager.patch
>
>
> HRegionInterface getHRegionConnection(final String hostname,
> final int port, final InetSocketAddress isa, final boolean master)
> throws IOException
> /////////////////////////
> String rsName = isa != null ? isa.toString() : Addressing
> .createHostAndPortStr(hostname, port);
>
> ////here,if isa is null, the Addressing created a address like "node41:60010"
> ////should use "isa.toString():new InetSocketAddress(hostname, port).toString();"
> ////instead of "Addressing.createHostAndPortStr(hostname, port);"
> server = this.servers.get(rsName);
> if (server == null) {
> // create a unique lock for this RS (if necessary)
> this.connectionLock.putIfAbsent(rsName, rsName);
> // get the RS lock
> synchronized (this.connectionLock.get(rsName)) {
> // do one more lookup in case we were stalled above
> server = this.servers.get(rsName);
> if (server == null) {
> try {
> if (clusterId.hasId()) {
> conf.set(HConstants.CLUSTER_ID, clusterId.getId());
> }
> // Only create isa when we need to.
> InetSocketAddress address = isa != null ? isa
> : new InetSocketAddress(hostname, port);
> // definitely a cache miss. establish an RPC for this RS
> server = (HRegionInterface) HBaseRPC.waitForProxy(
> serverInterfaceClass, HRegionInterface.VERSION, address,
> this.conf, this.maxRPCAttempts, this.rpcTimeout,
> this.rpcTimeout);
> this.servers.put(address.toString(), server);
>
> ////but here address.toString() send an address like "node41/10.61.2l.171:60010
> ////so this method can never get cached address and make client request very slow due to it's synchronized.
>
>
> } catch (RemoteException e) {
> LOG.warn("RemoteException connecting to RS", e);
> // Throw what the RemoteException was carrying.
> throw RemoteExceptionHandler.decodeRemoteException(e);
> }
> }
> }
> ///////////////////////
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4181) HConnectionManager can't find
cached HRegionInterface makes client very slow
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081396#comment-13081396 ]
Ted Yu commented on HBASE-4181:
-------------------------------
InetSocketAddress ctor involves DNS query. We should try not to call it.
The current code resorts to InetSocketAddress ctor under a cache miss.
Shall we modify the following code so that the key is in the same format as Addressing.createHostAndPortStr() ?
{code}
this.servers.put(address.toString(), server);
{code}
> HConnectionManager can't find cached HRegionInterface makes client very slow
> ----------------------------------------------------------------------------
>
> Key: HBASE-4181
> URL: https://issues.apache.org/jira/browse/HBASE-4181
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.92.0
> Reporter: Liu Jia
> Priority: Critical
> Labels: HConnectionManager
> Attachments: HConnectionManager.patch
>
>
> HRegionInterface getHRegionConnection(final String hostname,
> final int port, final InetSocketAddress isa, final boolean master)
> throws IOException
> /////////////////////////
> String rsName = isa != null ? isa.toString() : Addressing
> .createHostAndPortStr(hostname, port);
>
> ////here,if isa is null, the Addressing created a address like "node41:60010"
> ////should use "isa.toString():new InetSocketAddress(hostname, port).toString();"
> ////instead of "Addressing.createHostAndPortStr(hostname, port);"
> server = this.servers.get(rsName);
> if (server == null) {
> // create a unique lock for this RS (if necessary)
> this.connectionLock.putIfAbsent(rsName, rsName);
> // get the RS lock
> synchronized (this.connectionLock.get(rsName)) {
> // do one more lookup in case we were stalled above
> server = this.servers.get(rsName);
> if (server == null) {
> try {
> if (clusterId.hasId()) {
> conf.set(HConstants.CLUSTER_ID, clusterId.getId());
> }
> // Only create isa when we need to.
> InetSocketAddress address = isa != null ? isa
> : new InetSocketAddress(hostname, port);
> // definitely a cache miss. establish an RPC for this RS
> server = (HRegionInterface) HBaseRPC.waitForProxy(
> serverInterfaceClass, HRegionInterface.VERSION, address,
> this.conf, this.maxRPCAttempts, this.rpcTimeout,
> this.rpcTimeout);
> this.servers.put(address.toString(), server);
>
> ////but here address.toString() send an address like "node41/10.61.2l.171:60010
> ////so this method can never get cached address and make client request very slow due to it's synchronized.
>
>
> } catch (RemoteException e) {
> LOG.warn("RemoteException connecting to RS", e);
> // Throw what the RemoteException was carrying.
> throw RemoteExceptionHandler.decodeRemoteException(e);
> }
> }
> }
> ///////////////////////
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4181) HConnectionManager can't find
cached HRegionInterface makes client very slow
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081643#comment-13081643 ]
jiraposter@reviews.apache.org commented on HBASE-4181:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1402/#review1343
-----------------------------------------------------------
Ship it!
- Ted
On 2011-08-09 09:41:48, Jia Liu wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/1402/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-08-09 09:41:48)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. HConnectionManager can't find cached HRegionInterface makes client very slow
bq. Addressing.createHostAndPortStr(hostname, port); //the Addressing created a address like "node41:60010"
bq. ......
bq. this.servers.put(address.toString(), server);
bq.
bq. //but here address.toString() send an address like "node41/10.61.2l.171:60010
bq.
bq.
bq. This addresses bug HBase-4181.
bq. https://issues.apache.org/jira/browse/HBase-4181
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 1155226
bq.
bq. Diff: https://reviews.apache.org/r/1402/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Tests passed locally.
bq.
bq.
bq. Thanks,
bq.
bq. Jia
bq.
bq.
> HConnectionManager can't find cached HRegionInterface makes client very slow
> ----------------------------------------------------------------------------
>
> Key: HBASE-4181
> URL: https://issues.apache.org/jira/browse/HBASE-4181
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.92.0
> Reporter: Liu Jia
> Priority: Critical
> Labels: HConnectionManager
> Attachments: HBASE-4181-trunk-v2.patch, HBASE-4181.patch, HConnectionManager.patch
>
>
> HRegionInterface getHRegionConnection(final String hostname,
> final int port, final InetSocketAddress isa, final boolean master)
> throws IOException
> /////////////////////////
> String rsName = isa != null ? isa.toString() : Addressing
> .createHostAndPortStr(hostname, port);
>
> ////here,if isa is null, the Addressing created a address like "node41:60010"
> ////should use "isa.toString():new InetSocketAddress(hostname, port).toString();"
> ////instead of "Addressing.createHostAndPortStr(hostname, port);"
> server = this.servers.get(rsName);
> if (server == null) {
> // create a unique lock for this RS (if necessary)
> this.connectionLock.putIfAbsent(rsName, rsName);
> // get the RS lock
> synchronized (this.connectionLock.get(rsName)) {
> // do one more lookup in case we were stalled above
> server = this.servers.get(rsName);
> if (server == null) {
> try {
> if (clusterId.hasId()) {
> conf.set(HConstants.CLUSTER_ID, clusterId.getId());
> }
> // Only create isa when we need to.
> InetSocketAddress address = isa != null ? isa
> : new InetSocketAddress(hostname, port);
> // definitely a cache miss. establish an RPC for this RS
> server = (HRegionInterface) HBaseRPC.waitForProxy(
> serverInterfaceClass, HRegionInterface.VERSION, address,
> this.conf, this.maxRPCAttempts, this.rpcTimeout,
> this.rpcTimeout);
> this.servers.put(address.toString(), server);
>
> ////but here address.toString() send an address like "node41/10.61.2l.171:60010
> ////so this method can never get cached address and make client request very slow due to it's synchronized.
>
>
> } catch (RemoteException e) {
> LOG.warn("RemoteException connecting to RS", e);
> // Throw what the RemoteException was carrying.
> throw RemoteExceptionHandler.decodeRemoteException(e);
> }
> }
> }
> ///////////////////////
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4181) HConnectionManager can't find cached
HRegionInterface which makes client very slow
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4181:
--------------------------
Summary: HConnectionManager can't find cached HRegionInterface which makes client very slow (was: HConnectionManager can't find cached HRegionInterface makes client very slow)
> HConnectionManager can't find cached HRegionInterface which makes client very slow
> ----------------------------------------------------------------------------------
>
> Key: HBASE-4181
> URL: https://issues.apache.org/jira/browse/HBASE-4181
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.92.0
> Reporter: Liu Jia
> Assignee: Liu Jia
> Priority: Critical
> Labels: HConnectionManager
> Fix For: 0.92.0
>
> Attachments: HBASE-4181-trunk-v2.patch, HBASE-4181-trunk-v3.patch, HBASE-4181.patch, HConnectionManager.patch
>
>
> HRegionInterface getHRegionConnection(final String hostname,
> final int port, final InetSocketAddress isa, final boolean master)
> throws IOException
> /////////////////////////
> String rsName = isa != null ? isa.toString() : Addressing
> .createHostAndPortStr(hostname, port);
>
> ////here,if isa is null, the Addressing created a address like "node41:60010"
> ////should use "isa.toString():new InetSocketAddress(hostname, port).toString();"
> ////instead of "Addressing.createHostAndPortStr(hostname, port);"
> server = this.servers.get(rsName);
> if (server == null) {
> // create a unique lock for this RS (if necessary)
> this.connectionLock.putIfAbsent(rsName, rsName);
> // get the RS lock
> synchronized (this.connectionLock.get(rsName)) {
> // do one more lookup in case we were stalled above
> server = this.servers.get(rsName);
> if (server == null) {
> try {
> if (clusterId.hasId()) {
> conf.set(HConstants.CLUSTER_ID, clusterId.getId());
> }
> // Only create isa when we need to.
> InetSocketAddress address = isa != null ? isa
> : new InetSocketAddress(hostname, port);
> // definitely a cache miss. establish an RPC for this RS
> server = (HRegionInterface) HBaseRPC.waitForProxy(
> serverInterfaceClass, HRegionInterface.VERSION, address,
> this.conf, this.maxRPCAttempts, this.rpcTimeout,
> this.rpcTimeout);
> this.servers.put(address.toString(), server);
>
> ////but here address.toString() send an address like "node41/10.61.2l.171:60010
> ////so this method can never get cached address and make client request very slow due to it's synchronized.
>
>
> } catch (RemoteException e) {
> LOG.warn("RemoteException connecting to RS", e);
> // Throw what the RemoteException was carrying.
> throw RemoteExceptionHandler.decodeRemoteException(e);
> }
> }
> }
> ///////////////////////
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4181) HConnectionManager can't find cached
HRegionInterface makes client very slow
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4181:
-------------------------
Resolution: Fixed
Fix Version/s: (was: 0.90.4)
0.92.0
Assignee: Liu Jia
Status: Resolved (was: Patch Available)
Committed to TRUNK. Thank you for the patch Liu Jia and Ted for review.
> HConnectionManager can't find cached HRegionInterface makes client very slow
> ----------------------------------------------------------------------------
>
> Key: HBASE-4181
> URL: https://issues.apache.org/jira/browse/HBASE-4181
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.92.0
> Reporter: Liu Jia
> Assignee: Liu Jia
> Priority: Critical
> Labels: HConnectionManager
> Fix For: 0.92.0
>
> Attachments: HBASE-4181-trunk-v2.patch, HBASE-4181-trunk-v3.patch, HBASE-4181.patch, HConnectionManager.patch
>
>
> HRegionInterface getHRegionConnection(final String hostname,
> final int port, final InetSocketAddress isa, final boolean master)
> throws IOException
> /////////////////////////
> String rsName = isa != null ? isa.toString() : Addressing
> .createHostAndPortStr(hostname, port);
>
> ////here,if isa is null, the Addressing created a address like "node41:60010"
> ////should use "isa.toString():new InetSocketAddress(hostname, port).toString();"
> ////instead of "Addressing.createHostAndPortStr(hostname, port);"
> server = this.servers.get(rsName);
> if (server == null) {
> // create a unique lock for this RS (if necessary)
> this.connectionLock.putIfAbsent(rsName, rsName);
> // get the RS lock
> synchronized (this.connectionLock.get(rsName)) {
> // do one more lookup in case we were stalled above
> server = this.servers.get(rsName);
> if (server == null) {
> try {
> if (clusterId.hasId()) {
> conf.set(HConstants.CLUSTER_ID, clusterId.getId());
> }
> // Only create isa when we need to.
> InetSocketAddress address = isa != null ? isa
> : new InetSocketAddress(hostname, port);
> // definitely a cache miss. establish an RPC for this RS
> server = (HRegionInterface) HBaseRPC.waitForProxy(
> serverInterfaceClass, HRegionInterface.VERSION, address,
> this.conf, this.maxRPCAttempts, this.rpcTimeout,
> this.rpcTimeout);
> this.servers.put(address.toString(), server);
>
> ////but here address.toString() send an address like "node41/10.61.2l.171:60010
> ////so this method can never get cached address and make client request very slow due to it's synchronized.
>
>
> } catch (RemoteException e) {
> LOG.warn("RemoteException connecting to RS", e);
> // Throw what the RemoteException was carrying.
> throw RemoteExceptionHandler.decodeRemoteException(e);
> }
> }
> }
> ///////////////////////
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4181) HConnectionManager can't find cached
HRegionInterface makes client very slow
Posted by "Liu Jia (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Liu Jia updated HBASE-4181:
---------------------------
Description:
HRegionInterface getHRegionConnection(final String hostname,
final int port, final InetSocketAddress isa, final boolean master)
throws IOException
/////////////////////////
String rsName = isa != null ? isa.toString() : Addressing
.createHostAndPortStr(hostname, port);
////here,if isa is null, the Addressing created a address like "node41:60010"
////should use "isa.toString():new InetSocketAddress(hostname, port).toString();"
////instead of "Addressing.createHostAndPortStr(hostname, port);"
server = this.servers.get(rsName);
if (server == null) {
// create a unique lock for this RS (if necessary)
this.connectionLock.putIfAbsent(rsName, rsName);
// get the RS lock
synchronized (this.connectionLock.get(rsName)) {
// do one more lookup in case we were stalled above
server = this.servers.get(rsName);
if (server == null) {
try {
if (clusterId.hasId()) {
conf.set(HConstants.CLUSTER_ID, clusterId.getId());
}
// Only create isa when we need to.
InetSocketAddress address = isa != null ? isa
: new InetSocketAddress(hostname, port);
// definitely a cache miss. establish an RPC for this RS
server = (HRegionInterface) HBaseRPC.waitForProxy(
serverInterfaceClass, HRegionInterface.VERSION, address,
this.conf, this.maxRPCAttempts, this.rpcTimeout,
this.rpcTimeout);
this.servers.put(address.toString(), server);
////but here address.toString() send an address like "node41/10.61.2l.171:60010
////so this method can never get cached address and make client request very slow due to it's synchronized.
} catch (RemoteException e) {
LOG.warn("RemoteException connecting to RS", e);
// Throw what the RemoteException was carrying.
throw RemoteExceptionHandler.decodeRemoteException(e);
}
}
}
///////////////////////
was:
HRegionInterface getHRegionConnection(final String hostname,
final int port, final InetSocketAddress isa, final boolean master)
throws IOException
/////////////////////////
String rsName = isa != null ? isa.toString() : Addressing
.createHostAndPortStr(hostname, port); ////here,if isa is null, the Addressing created a address like "node41:60010"
////isa.toString():new InetSocketAddress(hostname, port).toString();
////instead of Addressing.createHostAndPortStr(hostname, port);
server = this.servers.get(rsName);
if (server == null) {
// create a unique lock for this RS (if necessary)
this.connectionLock.putIfAbsent(rsName, rsName);
// get the RS lock
synchronized (this.connectionLock.get(rsName)) {
// do one more lookup in case we were stalled above
server = this.servers.get(rsName);
if (server == null) {
try {
if (clusterId.hasId()) {
conf.set(HConstants.CLUSTER_ID, clusterId.getId());
}
// Only create isa when we need to.
InetSocketAddress address = isa != null ? isa
: new InetSocketAddress(hostname, port);
// definitely a cache miss. establish an RPC for this RS
server = (HRegionInterface) HBaseRPC.waitForProxy(
serverInterfaceClass, HRegionInterface.VERSION, address,
this.conf, this.maxRPCAttempts, this.rpcTimeout,
this.rpcTimeout);
this.servers.put(address.toString(), server); ////but here address.toString() send an address like "node41/10.61.2l.171:60010"
////so this method can never get cached address and make client request very slow ////due to it's synchronized.
} catch (RemoteException e) {
LOG.warn("RemoteException connecting to RS", e);
// Throw what the RemoteException was carrying.
throw RemoteExceptionHandler.decodeRemoteException(e);
}
}
}
///////////////////////
> HConnectionManager can't find cached HRegionInterface makes client very slow
> ----------------------------------------------------------------------------
>
> Key: HBASE-4181
> URL: https://issues.apache.org/jira/browse/HBASE-4181
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.92.0
> Reporter: Liu Jia
> Priority: Critical
> Labels: HConnectionManager
>
> HRegionInterface getHRegionConnection(final String hostname,
> final int port, final InetSocketAddress isa, final boolean master)
> throws IOException
> /////////////////////////
> String rsName = isa != null ? isa.toString() : Addressing
> .createHostAndPortStr(hostname, port);
>
> ////here,if isa is null, the Addressing created a address like "node41:60010"
> ////should use "isa.toString():new InetSocketAddress(hostname, port).toString();"
> ////instead of "Addressing.createHostAndPortStr(hostname, port);"
> server = this.servers.get(rsName);
> if (server == null) {
> // create a unique lock for this RS (if necessary)
> this.connectionLock.putIfAbsent(rsName, rsName);
> // get the RS lock
> synchronized (this.connectionLock.get(rsName)) {
> // do one more lookup in case we were stalled above
> server = this.servers.get(rsName);
> if (server == null) {
> try {
> if (clusterId.hasId()) {
> conf.set(HConstants.CLUSTER_ID, clusterId.getId());
> }
> // Only create isa when we need to.
> InetSocketAddress address = isa != null ? isa
> : new InetSocketAddress(hostname, port);
> // definitely a cache miss. establish an RPC for this RS
> server = (HRegionInterface) HBaseRPC.waitForProxy(
> serverInterfaceClass, HRegionInterface.VERSION, address,
> this.conf, this.maxRPCAttempts, this.rpcTimeout,
> this.rpcTimeout);
> this.servers.put(address.toString(), server);
>
> ////but here address.toString() send an address like "node41/10.61.2l.171:60010
> ////so this method can never get cached address and make client request very slow due to it's synchronized.
>
>
> } catch (RemoteException e) {
> LOG.warn("RemoteException connecting to RS", e);
> // Throw what the RemoteException was carrying.
> throw RemoteExceptionHandler.decodeRemoteException(e);
> }
> }
> }
> ///////////////////////
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4181) HConnectionManager can't find
cached HRegionInterface makes client very slow
Posted by "sinfox (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081389#comment-13081389 ]
sinfox commented on HBASE-4181:
-------------------------------
I encounter the same problem .
Thanks to this patch, it is effective.
> HConnectionManager can't find cached HRegionInterface makes client very slow
> ----------------------------------------------------------------------------
>
> Key: HBASE-4181
> URL: https://issues.apache.org/jira/browse/HBASE-4181
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.92.0
> Reporter: Liu Jia
> Priority: Critical
> Labels: HConnectionManager
> Attachments: HConnectionManager.patch
>
>
> HRegionInterface getHRegionConnection(final String hostname,
> final int port, final InetSocketAddress isa, final boolean master)
> throws IOException
> /////////////////////////
> String rsName = isa != null ? isa.toString() : Addressing
> .createHostAndPortStr(hostname, port);
>
> ////here,if isa is null, the Addressing created a address like "node41:60010"
> ////should use "isa.toString():new InetSocketAddress(hostname, port).toString();"
> ////instead of "Addressing.createHostAndPortStr(hostname, port);"
> server = this.servers.get(rsName);
> if (server == null) {
> // create a unique lock for this RS (if necessary)
> this.connectionLock.putIfAbsent(rsName, rsName);
> // get the RS lock
> synchronized (this.connectionLock.get(rsName)) {
> // do one more lookup in case we were stalled above
> server = this.servers.get(rsName);
> if (server == null) {
> try {
> if (clusterId.hasId()) {
> conf.set(HConstants.CLUSTER_ID, clusterId.getId());
> }
> // Only create isa when we need to.
> InetSocketAddress address = isa != null ? isa
> : new InetSocketAddress(hostname, port);
> // definitely a cache miss. establish an RPC for this RS
> server = (HRegionInterface) HBaseRPC.waitForProxy(
> serverInterfaceClass, HRegionInterface.VERSION, address,
> this.conf, this.maxRPCAttempts, this.rpcTimeout,
> this.rpcTimeout);
> this.servers.put(address.toString(), server);
>
> ////but here address.toString() send an address like "node41/10.61.2l.171:60010
> ////so this method can never get cached address and make client request very slow due to it's synchronized.
>
>
> } catch (RemoteException e) {
> LOG.warn("RemoteException connecting to RS", e);
> // Throw what the RemoteException was carrying.
> throw RemoteExceptionHandler.decodeRemoteException(e);
> }
> }
> }
> ///////////////////////
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4181) HConnectionManager can't find cached
HRegionInterface makes client very slow
Posted by "Liu Jia (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Liu Jia updated HBASE-4181:
---------------------------
Fix Version/s: 0.90.4
Release Note: remove DNS query part from getHRegionConnection()
Hadoop Flags: [Reviewed]
Status: Patch Available (was: Open)
> HConnectionManager can't find cached HRegionInterface makes client very slow
> ----------------------------------------------------------------------------
>
> Key: HBASE-4181
> URL: https://issues.apache.org/jira/browse/HBASE-4181
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.92.0
> Reporter: Liu Jia
> Priority: Critical
> Labels: HConnectionManager
> Fix For: 0.90.4
>
> Attachments: HBASE-4181-trunk-v2.patch, HBASE-4181-trunk-v3.patch, HBASE-4181.patch, HConnectionManager.patch
>
>
> HRegionInterface getHRegionConnection(final String hostname,
> final int port, final InetSocketAddress isa, final boolean master)
> throws IOException
> /////////////////////////
> String rsName = isa != null ? isa.toString() : Addressing
> .createHostAndPortStr(hostname, port);
>
> ////here,if isa is null, the Addressing created a address like "node41:60010"
> ////should use "isa.toString():new InetSocketAddress(hostname, port).toString();"
> ////instead of "Addressing.createHostAndPortStr(hostname, port);"
> server = this.servers.get(rsName);
> if (server == null) {
> // create a unique lock for this RS (if necessary)
> this.connectionLock.putIfAbsent(rsName, rsName);
> // get the RS lock
> synchronized (this.connectionLock.get(rsName)) {
> // do one more lookup in case we were stalled above
> server = this.servers.get(rsName);
> if (server == null) {
> try {
> if (clusterId.hasId()) {
> conf.set(HConstants.CLUSTER_ID, clusterId.getId());
> }
> // Only create isa when we need to.
> InetSocketAddress address = isa != null ? isa
> : new InetSocketAddress(hostname, port);
> // definitely a cache miss. establish an RPC for this RS
> server = (HRegionInterface) HBaseRPC.waitForProxy(
> serverInterfaceClass, HRegionInterface.VERSION, address,
> this.conf, this.maxRPCAttempts, this.rpcTimeout,
> this.rpcTimeout);
> this.servers.put(address.toString(), server);
>
> ////but here address.toString() send an address like "node41/10.61.2l.171:60010
> ////so this method can never get cached address and make client request very slow due to it's synchronized.
>
>
> } catch (RemoteException e) {
> LOG.warn("RemoteException connecting to RS", e);
> // Throw what the RemoteException was carrying.
> throw RemoteExceptionHandler.decodeRemoteException(e);
> }
> }
> }
> ///////////////////////
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4181) HConnectionManager can't find cached
HRegionInterface makes client very slow
Posted by "Liu Jia (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Liu Jia updated HBASE-4181:
---------------------------
Attachment: HBASE-4181-trunk-v3.patch
> HConnectionManager can't find cached HRegionInterface makes client very slow
> ----------------------------------------------------------------------------
>
> Key: HBASE-4181
> URL: https://issues.apache.org/jira/browse/HBASE-4181
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.92.0
> Reporter: Liu Jia
> Priority: Critical
> Labels: HConnectionManager
> Attachments: HBASE-4181-trunk-v2.patch, HBASE-4181-trunk-v3.patch, HBASE-4181.patch, HConnectionManager.patch
>
>
> HRegionInterface getHRegionConnection(final String hostname,
> final int port, final InetSocketAddress isa, final boolean master)
> throws IOException
> /////////////////////////
> String rsName = isa != null ? isa.toString() : Addressing
> .createHostAndPortStr(hostname, port);
>
> ////here,if isa is null, the Addressing created a address like "node41:60010"
> ////should use "isa.toString():new InetSocketAddress(hostname, port).toString();"
> ////instead of "Addressing.createHostAndPortStr(hostname, port);"
> server = this.servers.get(rsName);
> if (server == null) {
> // create a unique lock for this RS (if necessary)
> this.connectionLock.putIfAbsent(rsName, rsName);
> // get the RS lock
> synchronized (this.connectionLock.get(rsName)) {
> // do one more lookup in case we were stalled above
> server = this.servers.get(rsName);
> if (server == null) {
> try {
> if (clusterId.hasId()) {
> conf.set(HConstants.CLUSTER_ID, clusterId.getId());
> }
> // Only create isa when we need to.
> InetSocketAddress address = isa != null ? isa
> : new InetSocketAddress(hostname, port);
> // definitely a cache miss. establish an RPC for this RS
> server = (HRegionInterface) HBaseRPC.waitForProxy(
> serverInterfaceClass, HRegionInterface.VERSION, address,
> this.conf, this.maxRPCAttempts, this.rpcTimeout,
> this.rpcTimeout);
> this.servers.put(address.toString(), server);
>
> ////but here address.toString() send an address like "node41/10.61.2l.171:60010
> ////so this method can never get cached address and make client request very slow due to it's synchronized.
>
>
> } catch (RemoteException e) {
> LOG.warn("RemoteException connecting to RS", e);
> // Throw what the RemoteException was carrying.
> throw RemoteExceptionHandler.decodeRemoteException(e);
> }
> }
> }
> ///////////////////////
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4181) HConnectionManager can't find cached
HRegionInterface makes client very slow
Posted by "Liu Jia (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Liu Jia updated HBASE-4181:
---------------------------
Attachment: HBASE-4181-trunk-v2.patch
> HConnectionManager can't find cached HRegionInterface makes client very slow
> ----------------------------------------------------------------------------
>
> Key: HBASE-4181
> URL: https://issues.apache.org/jira/browse/HBASE-4181
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.92.0
> Reporter: Liu Jia
> Priority: Critical
> Labels: HConnectionManager
> Attachments: HBASE-4181-trunk-v2.patch, HBASE-4181.patch, HConnectionManager.patch
>
>
> HRegionInterface getHRegionConnection(final String hostname,
> final int port, final InetSocketAddress isa, final boolean master)
> throws IOException
> /////////////////////////
> String rsName = isa != null ? isa.toString() : Addressing
> .createHostAndPortStr(hostname, port);
>
> ////here,if isa is null, the Addressing created a address like "node41:60010"
> ////should use "isa.toString():new InetSocketAddress(hostname, port).toString();"
> ////instead of "Addressing.createHostAndPortStr(hostname, port);"
> server = this.servers.get(rsName);
> if (server == null) {
> // create a unique lock for this RS (if necessary)
> this.connectionLock.putIfAbsent(rsName, rsName);
> // get the RS lock
> synchronized (this.connectionLock.get(rsName)) {
> // do one more lookup in case we were stalled above
> server = this.servers.get(rsName);
> if (server == null) {
> try {
> if (clusterId.hasId()) {
> conf.set(HConstants.CLUSTER_ID, clusterId.getId());
> }
> // Only create isa when we need to.
> InetSocketAddress address = isa != null ? isa
> : new InetSocketAddress(hostname, port);
> // definitely a cache miss. establish an RPC for this RS
> server = (HRegionInterface) HBaseRPC.waitForProxy(
> serverInterfaceClass, HRegionInterface.VERSION, address,
> this.conf, this.maxRPCAttempts, this.rpcTimeout,
> this.rpcTimeout);
> this.servers.put(address.toString(), server);
>
> ////but here address.toString() send an address like "node41/10.61.2l.171:60010
> ////so this method can never get cached address and make client request very slow due to it's synchronized.
>
>
> } catch (RemoteException e) {
> LOG.warn("RemoteException connecting to RS", e);
> // Throw what the RemoteException was carrying.
> throw RemoteExceptionHandler.decodeRemoteException(e);
> }
> }
> }
> ///////////////////////
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4181) HConnectionManager can't find
cached HRegionInterface makes client very slow
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081472#comment-13081472 ]
jiraposter@reviews.apache.org commented on HBASE-4181:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1402/
-----------------------------------------------------------
(Updated 2011-08-09 06:41:43.389550)
Review request for hbase.
Changes
-------
to avoid DNS query.
Summary
-------
HConnectionManager can't find cached HRegionInterface makes client very slow
Addressing.createHostAndPortStr(hostname, port); //the Addressing created a address like "node41:60010"
......
this.servers.put(address.toString(), server);
//but here address.toString() send an address like "node41/10.61.2l.171:60010
This addresses bug HBase-4181.
https://issues.apache.org/jira/browse/HBase-4181
Diffs (updated)
-----
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 1155226
Diff: https://reviews.apache.org/r/1402/diff
Testing
-------
Tests passed locally.
Thanks,
Jia
> HConnectionManager can't find cached HRegionInterface makes client very slow
> ----------------------------------------------------------------------------
>
> Key: HBASE-4181
> URL: https://issues.apache.org/jira/browse/HBASE-4181
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.92.0
> Reporter: Liu Jia
> Priority: Critical
> Labels: HConnectionManager
> Attachments: HBASE-4181.patch, HConnectionManager.patch
>
>
> HRegionInterface getHRegionConnection(final String hostname,
> final int port, final InetSocketAddress isa, final boolean master)
> throws IOException
> /////////////////////////
> String rsName = isa != null ? isa.toString() : Addressing
> .createHostAndPortStr(hostname, port);
>
> ////here,if isa is null, the Addressing created a address like "node41:60010"
> ////should use "isa.toString():new InetSocketAddress(hostname, port).toString();"
> ////instead of "Addressing.createHostAndPortStr(hostname, port);"
> server = this.servers.get(rsName);
> if (server == null) {
> // create a unique lock for this RS (if necessary)
> this.connectionLock.putIfAbsent(rsName, rsName);
> // get the RS lock
> synchronized (this.connectionLock.get(rsName)) {
> // do one more lookup in case we were stalled above
> server = this.servers.get(rsName);
> if (server == null) {
> try {
> if (clusterId.hasId()) {
> conf.set(HConstants.CLUSTER_ID, clusterId.getId());
> }
> // Only create isa when we need to.
> InetSocketAddress address = isa != null ? isa
> : new InetSocketAddress(hostname, port);
> // definitely a cache miss. establish an RPC for this RS
> server = (HRegionInterface) HBaseRPC.waitForProxy(
> serverInterfaceClass, HRegionInterface.VERSION, address,
> this.conf, this.maxRPCAttempts, this.rpcTimeout,
> this.rpcTimeout);
> this.servers.put(address.toString(), server);
>
> ////but here address.toString() send an address like "node41/10.61.2l.171:60010
> ////so this method can never get cached address and make client request very slow due to it's synchronized.
>
>
> } catch (RemoteException e) {
> LOG.warn("RemoteException connecting to RS", e);
> // Throw what the RemoteException was carrying.
> throw RemoteExceptionHandler.decodeRemoteException(e);
> }
> }
> }
> ///////////////////////
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4181) HConnectionManager can't find
cached HRegionInterface makes client very slow
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081441#comment-13081441 ]
jiraposter@reviews.apache.org commented on HBASE-4181:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1402/
-----------------------------------------------------------
Review request for hbase.
Summary
-------
HConnectionManager can't find cached HRegionInterface makes client very slow
Addressing.createHostAndPortStr(hostname, port); //the Addressing created a address like "node41:60010"
......
this.servers.put(address.toString(), server);
//but here address.toString() send an address like "node41/10.61.2l.171:60010
This addresses bug HBase-4181.
https://issues.apache.org/jira/browse/HBase-4181
Diffs
-----
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 1155206
Diff: https://reviews.apache.org/r/1402/diff
Testing
-------
Tests passed locally.
Thanks,
Jia
> HConnectionManager can't find cached HRegionInterface makes client very slow
> ----------------------------------------------------------------------------
>
> Key: HBASE-4181
> URL: https://issues.apache.org/jira/browse/HBASE-4181
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.92.0
> Reporter: Liu Jia
> Priority: Critical
> Labels: HConnectionManager
> Attachments: HConnectionManager.patch
>
>
> HRegionInterface getHRegionConnection(final String hostname,
> final int port, final InetSocketAddress isa, final boolean master)
> throws IOException
> /////////////////////////
> String rsName = isa != null ? isa.toString() : Addressing
> .createHostAndPortStr(hostname, port);
>
> ////here,if isa is null, the Addressing created a address like "node41:60010"
> ////should use "isa.toString():new InetSocketAddress(hostname, port).toString();"
> ////instead of "Addressing.createHostAndPortStr(hostname, port);"
> server = this.servers.get(rsName);
> if (server == null) {
> // create a unique lock for this RS (if necessary)
> this.connectionLock.putIfAbsent(rsName, rsName);
> // get the RS lock
> synchronized (this.connectionLock.get(rsName)) {
> // do one more lookup in case we were stalled above
> server = this.servers.get(rsName);
> if (server == null) {
> try {
> if (clusterId.hasId()) {
> conf.set(HConstants.CLUSTER_ID, clusterId.getId());
> }
> // Only create isa when we need to.
> InetSocketAddress address = isa != null ? isa
> : new InetSocketAddress(hostname, port);
> // definitely a cache miss. establish an RPC for this RS
> server = (HRegionInterface) HBaseRPC.waitForProxy(
> serverInterfaceClass, HRegionInterface.VERSION, address,
> this.conf, this.maxRPCAttempts, this.rpcTimeout,
> this.rpcTimeout);
> this.servers.put(address.toString(), server);
>
> ////but here address.toString() send an address like "node41/10.61.2l.171:60010
> ////so this method can never get cached address and make client request very slow due to it's synchronized.
>
>
> } catch (RemoteException e) {
> LOG.warn("RemoteException connecting to RS", e);
> // Throw what the RemoteException was carrying.
> throw RemoteExceptionHandler.decodeRemoteException(e);
> }
> }
> }
> ///////////////////////
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4181) HConnectionManager can't find cached
HRegionInterface makes client very slow
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4181:
--------------------------
Affects Version/s: (was: 0.90.4)
The cited code doesn't apply to 0.90 branch.
> HConnectionManager can't find cached HRegionInterface makes client very slow
> ----------------------------------------------------------------------------
>
> Key: HBASE-4181
> URL: https://issues.apache.org/jira/browse/HBASE-4181
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.92.0
> Reporter: Liu Jia
> Assignee: Liu Jia
> Priority: Critical
> Labels: HConnectionManager
> Fix For: 0.92.0
>
> Attachments: HBASE-4181-trunk-v2.patch, HBASE-4181-trunk-v3.patch, HBASE-4181.patch, HConnectionManager.patch
>
>
> HRegionInterface getHRegionConnection(final String hostname,
> final int port, final InetSocketAddress isa, final boolean master)
> throws IOException
> /////////////////////////
> String rsName = isa != null ? isa.toString() : Addressing
> .createHostAndPortStr(hostname, port);
>
> ////here,if isa is null, the Addressing created a address like "node41:60010"
> ////should use "isa.toString():new InetSocketAddress(hostname, port).toString();"
> ////instead of "Addressing.createHostAndPortStr(hostname, port);"
> server = this.servers.get(rsName);
> if (server == null) {
> // create a unique lock for this RS (if necessary)
> this.connectionLock.putIfAbsent(rsName, rsName);
> // get the RS lock
> synchronized (this.connectionLock.get(rsName)) {
> // do one more lookup in case we were stalled above
> server = this.servers.get(rsName);
> if (server == null) {
> try {
> if (clusterId.hasId()) {
> conf.set(HConstants.CLUSTER_ID, clusterId.getId());
> }
> // Only create isa when we need to.
> InetSocketAddress address = isa != null ? isa
> : new InetSocketAddress(hostname, port);
> // definitely a cache miss. establish an RPC for this RS
> server = (HRegionInterface) HBaseRPC.waitForProxy(
> serverInterfaceClass, HRegionInterface.VERSION, address,
> this.conf, this.maxRPCAttempts, this.rpcTimeout,
> this.rpcTimeout);
> this.servers.put(address.toString(), server);
>
> ////but here address.toString() send an address like "node41/10.61.2l.171:60010
> ////so this method can never get cached address and make client request very slow due to it's synchronized.
>
>
> } catch (RemoteException e) {
> LOG.warn("RemoteException connecting to RS", e);
> // Throw what the RemoteException was carrying.
> throw RemoteExceptionHandler.decodeRemoteException(e);
> }
> }
> }
> ///////////////////////
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4181) HConnectionManager can't find
cached HRegionInterface makes client very slow
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081456#comment-13081456 ]
jiraposter@reviews.apache.org commented on HBASE-4181:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1402/
-----------------------------------------------------------
(Updated 2011-08-09 05:53:25.766068)
Review request for hbase.
Summary (updated)
-------
HConnectionManager can't find cached HRegionInterface makes client very slow
Addressing.createHostAndPortStr(hostname, port); //the Addressing created a address like "node41:60010"
......
this.servers.put(address.toString(), server);
//but here address.toString() send an address like "node41/10.61.2l.171:60010
This addresses bug HBase-4181.
https://issues.apache.org/jira/browse/HBase-4181
Diffs
-----
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 1155206
Diff: https://reviews.apache.org/r/1402/diff
Testing
-------
Tests passed locally.
Thanks,
Jia
> HConnectionManager can't find cached HRegionInterface makes client very slow
> ----------------------------------------------------------------------------
>
> Key: HBASE-4181
> URL: https://issues.apache.org/jira/browse/HBASE-4181
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.92.0
> Reporter: Liu Jia
> Priority: Critical
> Labels: HConnectionManager
> Attachments: HConnectionManager.patch
>
>
> HRegionInterface getHRegionConnection(final String hostname,
> final int port, final InetSocketAddress isa, final boolean master)
> throws IOException
> /////////////////////////
> String rsName = isa != null ? isa.toString() : Addressing
> .createHostAndPortStr(hostname, port);
>
> ////here,if isa is null, the Addressing created a address like "node41:60010"
> ////should use "isa.toString():new InetSocketAddress(hostname, port).toString();"
> ////instead of "Addressing.createHostAndPortStr(hostname, port);"
> server = this.servers.get(rsName);
> if (server == null) {
> // create a unique lock for this RS (if necessary)
> this.connectionLock.putIfAbsent(rsName, rsName);
> // get the RS lock
> synchronized (this.connectionLock.get(rsName)) {
> // do one more lookup in case we were stalled above
> server = this.servers.get(rsName);
> if (server == null) {
> try {
> if (clusterId.hasId()) {
> conf.set(HConstants.CLUSTER_ID, clusterId.getId());
> }
> // Only create isa when we need to.
> InetSocketAddress address = isa != null ? isa
> : new InetSocketAddress(hostname, port);
> // definitely a cache miss. establish an RPC for this RS
> server = (HRegionInterface) HBaseRPC.waitForProxy(
> serverInterfaceClass, HRegionInterface.VERSION, address,
> this.conf, this.maxRPCAttempts, this.rpcTimeout,
> this.rpcTimeout);
> this.servers.put(address.toString(), server);
>
> ////but here address.toString() send an address like "node41/10.61.2l.171:60010
> ////so this method can never get cached address and make client request very slow due to it's synchronized.
>
>
> } catch (RemoteException e) {
> LOG.warn("RemoteException connecting to RS", e);
> // Throw what the RemoteException was carrying.
> throw RemoteExceptionHandler.decodeRemoteException(e);
> }
> }
> }
> ///////////////////////
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4181) HConnectionManager can't find cached
HRegionInterface makes client very slow
Posted by "Liu Jia (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Liu Jia updated HBASE-4181:
---------------------------
Release Note: Use the host and port string as the key of the Map<String, HRegionInterface> servers to avoid cache misses caused by DNS resolution.
> HConnectionManager can't find cached HRegionInterface makes client very slow
> ----------------------------------------------------------------------------
>
> Key: HBASE-4181
> URL: https://issues.apache.org/jira/browse/HBASE-4181
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.92.0
> Reporter: Liu Jia
> Priority: Critical
> Labels: HConnectionManager
> Fix For: 0.90.4
>
> Attachments: HBASE-4181-trunk-v2.patch, HBASE-4181-trunk-v3.patch, HBASE-4181.patch, HConnectionManager.patch
>
>
> HRegionInterface getHRegionConnection(final String hostname,
> final int port, final InetSocketAddress isa, final boolean master)
> throws IOException
> /////////////////////////
> String rsName = isa != null ? isa.toString() : Addressing
> .createHostAndPortStr(hostname, port);
>
> ////here,if isa is null, the Addressing created a address like "node41:60010"
> ////should use "isa.toString():new InetSocketAddress(hostname, port).toString();"
> ////instead of "Addressing.createHostAndPortStr(hostname, port);"
> server = this.servers.get(rsName);
> if (server == null) {
> // create a unique lock for this RS (if necessary)
> this.connectionLock.putIfAbsent(rsName, rsName);
> // get the RS lock
> synchronized (this.connectionLock.get(rsName)) {
> // do one more lookup in case we were stalled above
> server = this.servers.get(rsName);
> if (server == null) {
> try {
> if (clusterId.hasId()) {
> conf.set(HConstants.CLUSTER_ID, clusterId.getId());
> }
> // Only create isa when we need to.
> InetSocketAddress address = isa != null ? isa
> : new InetSocketAddress(hostname, port);
> // definitely a cache miss. establish an RPC for this RS
> server = (HRegionInterface) HBaseRPC.waitForProxy(
> serverInterfaceClass, HRegionInterface.VERSION, address,
> this.conf, this.maxRPCAttempts, this.rpcTimeout,
> this.rpcTimeout);
> this.servers.put(address.toString(), server);
>
> ////but here address.toString() send an address like "node41/10.61.2l.171:60010
> ////so this method can never get cached address and make client request very slow due to it's synchronized.
>
>
> } catch (RemoteException e) {
> LOG.warn("RemoteException connecting to RS", e);
> // Throw what the RemoteException was carrying.
> throw RemoteExceptionHandler.decodeRemoteException(e);
> }
> }
> }
> ///////////////////////
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4181) HConnectionManager can't find
cached HRegionInterface makes client very slow
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081732#comment-13081732 ]
jiraposter@reviews.apache.org commented on HBASE-4181:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1402/#review1348
-----------------------------------------------------------
Ship it!
Nice fix. Thanks for doing it.
- Michael
On 2011-08-09 09:41:48, Jia Liu wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/1402/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-08-09 09:41:48)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. HConnectionManager can't find cached HRegionInterface makes client very slow
bq. Addressing.createHostAndPortStr(hostname, port); //the Addressing created a address like "node41:60010"
bq. ......
bq. this.servers.put(address.toString(), server);
bq.
bq. //but here address.toString() send an address like "node41/10.61.2l.171:60010
bq.
bq.
bq. This addresses bug HBase-4181.
bq. https://issues.apache.org/jira/browse/HBase-4181
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 1155226
bq.
bq. Diff: https://reviews.apache.org/r/1402/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Tests passed locally.
bq.
bq.
bq. Thanks,
bq.
bq. Jia
bq.
bq.
> HConnectionManager can't find cached HRegionInterface makes client very slow
> ----------------------------------------------------------------------------
>
> Key: HBASE-4181
> URL: https://issues.apache.org/jira/browse/HBASE-4181
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.92.0
> Reporter: Liu Jia
> Priority: Critical
> Labels: HConnectionManager
> Fix For: 0.90.4
>
> Attachments: HBASE-4181-trunk-v2.patch, HBASE-4181-trunk-v3.patch, HBASE-4181.patch, HConnectionManager.patch
>
>
> HRegionInterface getHRegionConnection(final String hostname,
> final int port, final InetSocketAddress isa, final boolean master)
> throws IOException
> /////////////////////////
> String rsName = isa != null ? isa.toString() : Addressing
> .createHostAndPortStr(hostname, port);
>
> ////here,if isa is null, the Addressing created a address like "node41:60010"
> ////should use "isa.toString():new InetSocketAddress(hostname, port).toString();"
> ////instead of "Addressing.createHostAndPortStr(hostname, port);"
> server = this.servers.get(rsName);
> if (server == null) {
> // create a unique lock for this RS (if necessary)
> this.connectionLock.putIfAbsent(rsName, rsName);
> // get the RS lock
> synchronized (this.connectionLock.get(rsName)) {
> // do one more lookup in case we were stalled above
> server = this.servers.get(rsName);
> if (server == null) {
> try {
> if (clusterId.hasId()) {
> conf.set(HConstants.CLUSTER_ID, clusterId.getId());
> }
> // Only create isa when we need to.
> InetSocketAddress address = isa != null ? isa
> : new InetSocketAddress(hostname, port);
> // definitely a cache miss. establish an RPC for this RS
> server = (HRegionInterface) HBaseRPC.waitForProxy(
> serverInterfaceClass, HRegionInterface.VERSION, address,
> this.conf, this.maxRPCAttempts, this.rpcTimeout,
> this.rpcTimeout);
> this.servers.put(address.toString(), server);
>
> ////but here address.toString() send an address like "node41/10.61.2l.171:60010
> ////so this method can never get cached address and make client request very slow due to it's synchronized.
>
>
> } catch (RemoteException e) {
> LOG.warn("RemoteException connecting to RS", e);
> // Throw what the RemoteException was carrying.
> throw RemoteExceptionHandler.decodeRemoteException(e);
> }
> }
> }
> ///////////////////////
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4181) HConnectionManager can't find cached
HRegionInterface makes client very slow
Posted by "Liu Jia (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Liu Jia updated HBASE-4181:
---------------------------
Attachment: HConnectionManager.patch
Fix this problem
> HConnectionManager can't find cached HRegionInterface makes client very slow
> ----------------------------------------------------------------------------
>
> Key: HBASE-4181
> URL: https://issues.apache.org/jira/browse/HBASE-4181
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.92.0
> Reporter: Liu Jia
> Priority: Critical
> Labels: HConnectionManager
> Attachments: HConnectionManager.patch
>
>
> HRegionInterface getHRegionConnection(final String hostname,
> final int port, final InetSocketAddress isa, final boolean master)
> throws IOException
> /////////////////////////
> String rsName = isa != null ? isa.toString() : Addressing
> .createHostAndPortStr(hostname, port);
>
> ////here,if isa is null, the Addressing created a address like "node41:60010"
> ////should use "isa.toString():new InetSocketAddress(hostname, port).toString();"
> ////instead of "Addressing.createHostAndPortStr(hostname, port);"
> server = this.servers.get(rsName);
> if (server == null) {
> // create a unique lock for this RS (if necessary)
> this.connectionLock.putIfAbsent(rsName, rsName);
> // get the RS lock
> synchronized (this.connectionLock.get(rsName)) {
> // do one more lookup in case we were stalled above
> server = this.servers.get(rsName);
> if (server == null) {
> try {
> if (clusterId.hasId()) {
> conf.set(HConstants.CLUSTER_ID, clusterId.getId());
> }
> // Only create isa when we need to.
> InetSocketAddress address = isa != null ? isa
> : new InetSocketAddress(hostname, port);
> // definitely a cache miss. establish an RPC for this RS
> server = (HRegionInterface) HBaseRPC.waitForProxy(
> serverInterfaceClass, HRegionInterface.VERSION, address,
> this.conf, this.maxRPCAttempts, this.rpcTimeout,
> this.rpcTimeout);
> this.servers.put(address.toString(), server);
>
> ////but here address.toString() send an address like "node41/10.61.2l.171:60010
> ////so this method can never get cached address and make client request very slow due to it's synchronized.
>
>
> } catch (RemoteException e) {
> LOG.warn("RemoteException connecting to RS", e);
> // Throw what the RemoteException was carrying.
> throw RemoteExceptionHandler.decodeRemoteException(e);
> }
> }
> }
> ///////////////////////
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4181) HConnectionManager can't find
cached HRegionInterface makes client very slow
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081529#comment-13081529 ]
jiraposter@reviews.apache.org commented on HBASE-4181:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1402/#review1339
-----------------------------------------------------------
Version 2 looks good.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
<https://reviews.apache.org/r/1402/#comment3028>
Line is too long.
Wrap so that it is under 80 chars long.
- Ted
On 2011-08-09 06:41:43, Jia Liu wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/1402/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-08-09 06:41:43)
bq.
bq.
bq. Review request for hbase.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. HConnectionManager can't find cached HRegionInterface makes client very slow
bq. Addressing.createHostAndPortStr(hostname, port); //the Addressing created a address like "node41:60010"
bq. ......
bq. this.servers.put(address.toString(), server);
bq.
bq. //but here address.toString() send an address like "node41/10.61.2l.171:60010
bq.
bq.
bq. This addresses bug HBase-4181.
bq. https://issues.apache.org/jira/browse/HBase-4181
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 1155226
bq.
bq. Diff: https://reviews.apache.org/r/1402/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Tests passed locally.
bq.
bq.
bq. Thanks,
bq.
bq. Jia
bq.
bq.
> HConnectionManager can't find cached HRegionInterface makes client very slow
> ----------------------------------------------------------------------------
>
> Key: HBASE-4181
> URL: https://issues.apache.org/jira/browse/HBASE-4181
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.92.0
> Reporter: Liu Jia
> Priority: Critical
> Labels: HConnectionManager
> Attachments: HBASE-4181-trunk-v2.patch, HBASE-4181.patch, HConnectionManager.patch
>
>
> HRegionInterface getHRegionConnection(final String hostname,
> final int port, final InetSocketAddress isa, final boolean master)
> throws IOException
> /////////////////////////
> String rsName = isa != null ? isa.toString() : Addressing
> .createHostAndPortStr(hostname, port);
>
> ////here,if isa is null, the Addressing created a address like "node41:60010"
> ////should use "isa.toString():new InetSocketAddress(hostname, port).toString();"
> ////instead of "Addressing.createHostAndPortStr(hostname, port);"
> server = this.servers.get(rsName);
> if (server == null) {
> // create a unique lock for this RS (if necessary)
> this.connectionLock.putIfAbsent(rsName, rsName);
> // get the RS lock
> synchronized (this.connectionLock.get(rsName)) {
> // do one more lookup in case we were stalled above
> server = this.servers.get(rsName);
> if (server == null) {
> try {
> if (clusterId.hasId()) {
> conf.set(HConstants.CLUSTER_ID, clusterId.getId());
> }
> // Only create isa when we need to.
> InetSocketAddress address = isa != null ? isa
> : new InetSocketAddress(hostname, port);
> // definitely a cache miss. establish an RPC for this RS
> server = (HRegionInterface) HBaseRPC.waitForProxy(
> serverInterfaceClass, HRegionInterface.VERSION, address,
> this.conf, this.maxRPCAttempts, this.rpcTimeout,
> this.rpcTimeout);
> this.servers.put(address.toString(), server);
>
> ////but here address.toString() send an address like "node41/10.61.2l.171:60010
> ////so this method can never get cached address and make client request very slow due to it's synchronized.
>
>
> } catch (RemoteException e) {
> LOG.warn("RemoteException connecting to RS", e);
> // Throw what the RemoteException was carrying.
> throw RemoteExceptionHandler.decodeRemoteException(e);
> }
> }
> }
> ///////////////////////
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4181) HConnectionManager can't find cached
HRegionInterface makes client very slow
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4181:
--------------------------
Release Note: Use the host and port string as the key of the Map<String, HRegionInterface> servers to avoid cache misses caused by different format of String returned by InetSocketAddress . (was: Use the host and port string as the key of the Map<String, HRegionInterface> servers to avoid cache misses caused by DNS resolution.)
> HConnectionManager can't find cached HRegionInterface makes client very slow
> ----------------------------------------------------------------------------
>
> Key: HBASE-4181
> URL: https://issues.apache.org/jira/browse/HBASE-4181
> Project: HBase
> Issue Type: Bug
> Components: client
> Affects Versions: 0.90.4, 0.92.0
> Reporter: Liu Jia
> Priority: Critical
> Labels: HConnectionManager
> Fix For: 0.90.4
>
> Attachments: HBASE-4181-trunk-v2.patch, HBASE-4181-trunk-v3.patch, HBASE-4181.patch, HConnectionManager.patch
>
>
> HRegionInterface getHRegionConnection(final String hostname,
> final int port, final InetSocketAddress isa, final boolean master)
> throws IOException
> /////////////////////////
> String rsName = isa != null ? isa.toString() : Addressing
> .createHostAndPortStr(hostname, port);
>
> ////here,if isa is null, the Addressing created a address like "node41:60010"
> ////should use "isa.toString():new InetSocketAddress(hostname, port).toString();"
> ////instead of "Addressing.createHostAndPortStr(hostname, port);"
> server = this.servers.get(rsName);
> if (server == null) {
> // create a unique lock for this RS (if necessary)
> this.connectionLock.putIfAbsent(rsName, rsName);
> // get the RS lock
> synchronized (this.connectionLock.get(rsName)) {
> // do one more lookup in case we were stalled above
> server = this.servers.get(rsName);
> if (server == null) {
> try {
> if (clusterId.hasId()) {
> conf.set(HConstants.CLUSTER_ID, clusterId.getId());
> }
> // Only create isa when we need to.
> InetSocketAddress address = isa != null ? isa
> : new InetSocketAddress(hostname, port);
> // definitely a cache miss. establish an RPC for this RS
> server = (HRegionInterface) HBaseRPC.waitForProxy(
> serverInterfaceClass, HRegionInterface.VERSION, address,
> this.conf, this.maxRPCAttempts, this.rpcTimeout,
> this.rpcTimeout);
> this.servers.put(address.toString(), server);
>
> ////but here address.toString() send an address like "node41/10.61.2l.171:60010
> ////so this method can never get cached address and make client request very slow due to it's synchronized.
>
>
> } catch (RemoteException e) {
> LOG.warn("RemoteException connecting to RS", e);
> // Throw what the RemoteException was carrying.
> throw RemoteExceptionHandler.decodeRemoteException(e);
> }
> }
> }
> ///////////////////////
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira