You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "ronghai.ma (Created) (JIRA)" <ji...@apache.org> on 2012/02/12 14:48:59 UTC

[jira] [Created] (HBASE-5388) tunning getCachedLocation method

tunning getCachedLocation method
--------------------------------

                 Key: HBASE-5388
                 URL: https://issues.apache.org/jira/browse/HBASE-5388
             Project: HBase
          Issue Type: Improvement
            Reporter: ronghai.ma


About 75% improvement。
1. Add a method in SoftValueSortedMap
public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
  return ((TreeMap) this.internalMap).lowerEntry(key);
}

2. Modify getCachedLocation 
Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
  if (tEntry != null) {
  HRegionLocation possibleRegion = tEntry.getValue();
  //other code
}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

[jira] [Updated] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zhihong Yu updated HBASE-5388:
------------------------------

    Attachment: 5388-v3.txt

Patch v3 aligns lowerValueByKey() with the rest of methods in SoftValueSortedMap.java
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207409#comment-13207409 ] 

Zhihong Yu commented on HBASE-5388:
-----------------------------------

TestSplitLogManager passed locally on MacBook with patch v6.

My script didn't find hanging test in build 956.

Will integrate patch v6 tomorrow if there is no further review.
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, 5388-v4.txt, 5388-v5.txt, 5388-v6.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207031#comment-13207031 ] 

stack commented on HBASE-5388:
------------------------------

I agree w/ Zhihong, that '+    if (!(this.internalMap instanceof NavigableMap)) {' block seems unnecessary.

Is the 'greatest' in this right?

{code}
+   * retrieves the value associated with the greatest key strictly less than
+   *  the given key, or null if there is no such key
{code}

Its unfortunate that SoftValueSortedMap sneaks back into HCM but I can live with it if no alternative (Lars' suggestion having SVSM implement NM didn't seem to bad but understand if it adds a bunch of boiler plate)


                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tune HConnectionManager#getCachedLocation method

Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207773#comment-13207773 ] 

Zhihong Yu commented on HBASE-5388:
-----------------------------------

Integrated to TRUNK.

Thanks for the patch Ronghai.

Thanks for the review, Lars.

@Ronghai:
Please resolve after committer integrates patch to source repository :-)
                
> Tune HConnectionManager#getCachedLocation method
> ------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, 5388-v4.txt, 5388-v5.txt, 5388-v6.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206674#comment-13206674 ] 

Lars Hofhansl commented on HBASE-5388:
--------------------------------------

Not too fond of this part:
{code}
-      SortedMap<byte [], HRegionLocation> tableLocations =
-        getTableLocations(tableName);
+      SoftValueSortedMap<byte [], HRegionLocation> tableLocations =
+        (SoftValueSortedMap) getTableLocations(tableName);
{code}
I did a patch earlier that removed all concrete references to SoftValueSortedMap, except for creation... This would be a step back.
Maybe SoftValueSortedMap should implement NavigableMap and just throw UnsupportedOperation on all other new methods, in that case SoftValueSortedMap could implement lowerEntry.

... Or maybe this would be overkill.

                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tune HConnectionManager#getCachedLocation method

Posted by "ronghai.ma (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13208160#comment-13208160 ] 

ronghai.ma commented on HBASE-5388:
-----------------------------------

@Zhihong
Thanks for the guide.

@Lars
Thanks for the review.
                
> Tune HConnectionManager#getCachedLocation method
> ------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, 5388-v4.txt, 5388-v5.txt, 5388-v6.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "ronghai.ma (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ronghai.ma updated HBASE-5388:
------------------------------

    Attachment: SoftValueSortedMap.java
                HConnectionManager.java

patch files
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: HConnectionManager.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "ronghai.ma (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207325#comment-13207325 ] 

ronghai.ma commented on HBASE-5388:
-----------------------------------

@Lars,Zhihong Yu
What about a new Class for the cache, there are too many synchronized and checkReferences in SoftValueSortedMap.

                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206641#comment-13206641 ] 

Hadoop QA commented on HBASE-5388:
----------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12514303/SoftValueSortedMap.java
  against trunk revision .

    +1 @author.  The patch does not contain any @author tags.

    -1 tests included.  The patch doesn't appear to include any new or modified tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    -1 patch.  The patch command could not apply the patch.

Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/945//console

This message is automatically generated.
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207401#comment-13207401 ] 

Hadoop QA commented on HBASE-5388:
----------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12514420/5388-v6.txt
  against trunk revision .

    +1 @author.  The patch does not contain any @author tags.

    -1 tests included.  The patch doesn't appear to include any new or modified tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    -1 javadoc.  The javadoc tool appears to have generated -136 warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    -1 findbugs.  The patch appears to introduce 157 new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit warnings.

     -1 core tests.  The patch failed these unit tests:
                       org.apache.hadoop.hbase.mapreduce.TestImportTsv
                  org.apache.hadoop.hbase.mapred.TestTableMapReduce
                  org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
                  org.apache.hadoop.hbase.master.TestSplitLogManager

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/956//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/956//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/956//console

This message is automatically generated.
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, 5388-v4.txt, 5388-v5.txt, 5388-v6.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206543#comment-13206543 ] 

Zhihong Yu commented on HBASE-5388:
-----------------------------------

@Ronghai:
I haven't seen your patch yet.
According to your description in the JIRA, I am guessing something similar to the following:
{code}
@@ -1067,9 +1067,12 @@
         return null;
       }

-      HRegionLocation rl = tableLocations.get(row);
-      if (rl != null) {
-        return rl;
+      Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
+      if (tEntry != null) {
+        HRegionLocation rl = tEntry.getValue();
+        if (rl != null) {
+          return rl;
+        }
       }
{code}
There is more code in getCachedLocation() following the above lines, revolving around tableLocations.headMap(row)
I wonder how you measured the performance boost - by instrumenting getCachedLocation() I assume ?
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207036#comment-13207036 ] 

Zhihong Yu commented on HBASE-5388:
-----------------------------------

The javadoc involving 'greatest' comes from javadoc for lowerEntry().

I will attach a new patch without the instanceof block.
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207338#comment-13207338 ] 

Lars Hofhansl commented on HBASE-5388:
--------------------------------------

@Ted: v3
I'd probably keep the name of possibleRegion (rather than rl), also keep this comment
{code}
-          // make sure that the end key is greater than the row we're looking
-          // for, otherwise the row actually belongs in the next region, not
-          // this one. the exception case is when the endkey is
-          // HConstants.EMPTY_START_ROW, signifying that the region we're
-          // checking is actually the last region in the table.
{code}
which is still valid.
Also is this needed:
{code}
+      if (value.get() == null) {
+        this.internalMap.remove(key);
+        return null;
+      }
{code}
? That is what checkReferences will do on the next access anyway, right?
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206561#comment-13206561 ] 

Zhihong Yu commented on HBASE-5388:
-----------------------------------

Please also publish performance numbers before and after the optimization.

Thanks
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207226#comment-13207226 ] 

Zhihong Yu commented on HBASE-5388:
-----------------------------------

The new method, lowerValueByKey(), isn't in NavigableMap.
If we restrict SoftValueSortedMap to only implementing NavigableMap, where should we place lowerValueByKey() ?
I assume we may find other place(s) in our code which can reuse lowerValueByKey().
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "ronghai.ma (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206691#comment-13206691 ] 

ronghai.ma commented on HBASE-5388:
-----------------------------------

@Lars Hofhansl
I prefer to make code short, and let the instance thin.
If SoftValueSortedMap implements NavigableMap, there will be more infomation in method_info and interfaces, and the interface method reference.

@Zhihong Yu
Thank you.
Kill the code which is not usefull.
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "ronghai.ma (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206579#comment-13206579 ] 

ronghai.ma commented on HBASE-5388:
-----------------------------------

CPU:IA32
JVM OPTIONS:-server
cached regions:60000
region row: 50000th region
prewarm each method: 1,000,000 times
run each method: 1,000,000 times
Time Usage:
By headMap(),lastKey(),get():1000ms
By lowerEntry(K):600ms

                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: HConnectionManager.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207413#comment-13207413 ] 

Zhihong Yu commented on HBASE-5388:
-----------------------------------

@Ronghai:
Do you think patch v6 is good to go ?

Thanks
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, 5388-v4.txt, 5388-v5.txt, 5388-v6.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HBASE-5388) tunning getCachedLocation method

Posted by "ronghai.ma (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ronghai.ma updated HBASE-5388:
------------------------------

                 Tags: tunning getCachedLocation  method
               Labels: patch  (was: )
    Affects Version/s: 0.90.5
         Release Note: 
About 75% improvement。
1. Add a method in SoftValueSortedMap
public synchronized <K, V> Entry<K, V> lowerEntry(K key) { return ((TreeMap) this.internalMap).lowerEntry(key); }

2. Modify getCachedLocation 
Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
if (tEntry != null) { HRegionLocation possibleRegion = tEntry.getValue(); //other code }
               Status: Patch Available  (was: Open)

About 75% improvement。
                
> tunning getCachedLocation method
> --------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>              Labels: patch
>
> About 75% improvement。
> 1. Add a method in SoftValueSortedMap
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> 2. Modify getCachedLocation 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206653#comment-13206653 ] 

Zhihong Yu commented on HBASE-5388:
-----------------------------------

@Ronghai:
Since this is an improvement, we should start from HBase TRUNK first.

Please generate patch from the root of your workspace by typing:
{code}
svn diff
{code}
Also, refer to HBASE-3678 for Eclipse formatter which takes care of formatting.

Keep a space between if and left parenthesis, a space between right parenthesis and left curly brace.
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tune HConnectionManager#getCachedLocation method

Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13208920#comment-13208920 ] 

Hudson commented on HBASE-5388:
-------------------------------

Integrated in HBase-TRUNK-security #112 (See [https://builds.apache.org/job/HBase-TRUNK-security/112/])
    HBASE-5388 Tune HConnectionManager#getCachedLocation method (Ronghai Ma) (Revision 1243994)

     Result = SUCCESS
tedyu : 
Files : 
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/SoftValueSortedMap.java

                
> Tune HConnectionManager#getCachedLocation method
> ------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, 5388-v4.txt, 5388-v5.txt, 5388-v6.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207310#comment-13207310 ] 

Zhihong Yu commented on HBASE-5388:
-----------------------------------

@Lars:
Can you clarify which patch you favor: v2 or v3 ?
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206686#comment-13206686 ] 

Zhihong Yu commented on HBASE-5388:
-----------------------------------

It wouldn't be an overkill if you have strong belief.
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zhihong Yu updated HBASE-5388:
------------------------------

    Attachment: 5388-v6.txt

Patch v6 adopts type-safety changes from Lars' patch.
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, 5388-v4.txt, 5388-v5.txt, 5388-v6.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207365#comment-13207365 ] 

Lars Hofhansl commented on HBASE-5388:
--------------------------------------

+1 on v6
Although I still think that 
{code}
+      if (value.get() == null) {
+        this.internalMap.remove(key);
+        return null;
+      }
{code}
is not needed.
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, 5388-v4.txt, 5388-v5.txt, 5388-v6.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207360#comment-13207360 ] 

Lars Hofhansl commented on HBASE-5388:
--------------------------------------

@rongmai: How's that? This has nothing to do with split regions.
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, 5388-v4.txt, 5388-v5.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "ronghai.ma (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207412#comment-13207412 ] 

ronghai.ma commented on HBASE-5388:
-----------------------------------

@Lars
If we change(put,remove) internalMap, we use synchronized for thread safe.

@Zhihong Yu
Scenario: There is a large scale of regions cached(more than 120,000), and using HConnectionImplementation#processBatch with a list(more than 20) concurrently.
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, 5388-v4.txt, 5388-v5.txt, 5388-v6.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zhihong Yu updated HBASE-5388:
------------------------------

    Attachment: 5388-v2.txt

Patch v2 is adapted from the files Ronghai attached.
I think the following check in SoftValueSortedMap.java isn't needed:
{code}
    if (!(this.internalMap instanceof NavigableMap)) {
{code}
because the two SoftValueSortedMap ctor's which don't instantiate TreeMap are private.

I included the if block for completeness.
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Assigned] (HBASE-5388) tunning getCachedLocation method

Posted by "Zhihong Yu (Assigned) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zhihong Yu reassigned HBASE-5388:
---------------------------------

    Assignee: ronghai.ma
    
> tunning getCachedLocation method
> --------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>
> About 75% improvement in execution time.
> 1. Add a method in SoftValueSortedMap
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207364#comment-13207364 ] 

Zhihong Yu commented on HBASE-5388:
-----------------------------------

@Lars:
What do you think of patch v6 ?
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, 5388-v4.txt, 5388-v5.txt, 5388-v6.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207375#comment-13207375 ] 

Zhihong Yu commented on HBASE-5388:
-----------------------------------

I prefer keeping the remove() call.
Isn't it nice that stale reference gets cleared early ?
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, 5388-v4.txt, 5388-v5.txt, 5388-v6.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "ronghai.ma (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206556#comment-13206556 ] 

ronghai.ma commented on HBASE-5388:
-----------------------------------

I will upload the patch latter, many meetings,take a nap.
I reviewed the code, and changed some code.

{code}
//SoftValueSortedMap#lowerKey
public synchronized K lowerKey(K key) {
  checkReferences();    
  return ((TreeMap<K, SoftValue<K,V>>)this.internalMap).lowerKey(key);
}
//SoftValueSortedMap#getLower
public synchronized V getLower(K key) {
  checkReferences(); 
  Map.Entry<K,SoftValue<K,V>> entry = ((TreeMap<K, SoftValue<K,V>>)this.internalMap).lowerEntry(key);
  if(entry==null){
    return null;
  }
  SoftValue<K,V> value=entry.getValue();
  if(value==null){
    return null;
  } 
  if (value.get() == null) {
    this.internalMap.remove(key);
    return null;
  }
  return value.get();
}
{code}
{code}
//HConnectionManager#HConnectionImplementation#getCachedLocation
HRegionLocation rl = tableLocations.get(row);
if (rl != null) {
  return rl;
}
rl = tableLocations.getLower(row);
if(r1==null){
  return null;
}
byte[] endKey = rl.getRegionInfo().getEndKey();
if(Bytes.equals(endKey,HConstants.EMPTY_END_ROW)||
  KeyValue.getRowComparator(tableName).compareRows(endKey, 0, endKey.length, row, 0, row.length) > 0){
  return rl;
}
{code}
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) tunning getCachedLocation method

Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206417#comment-13206417 ] 

Zhihong Yu commented on HBASE-5388:
-----------------------------------

@Ronghai:
Can you upload a patch ?
The improvement is measured in execution time, right ?
                
> tunning getCachedLocation method
> --------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: ronghai.ma
>
> About 75% improvement。
> 1. Add a method in SoftValueSortedMap
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> 2. Modify getCachedLocation 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "ronghai.ma (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207257#comment-13207257 ] 

ronghai.ma commented on HBASE-5388:
-----------------------------------

rename it lower()
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "ronghai.ma (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206581#comment-13206581 ] 

ronghai.ma commented on HBASE-5388:
-----------------------------------

@Lars hofhansl
thank you.
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: HConnectionManager.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206563#comment-13206563 ] 

Lars Hofhansl commented on HBASE-5388:
--------------------------------------

NavigableMap has lowerKey and lowerEntry, we should use the same naming scheme.
(Ideally SoftValueSortedMap should implement NavigableMap. But that is a different story.)

This whole block:
{code}
      SortedMap<byte[], HRegionLocation> matchingRegions =
        tableLocations.headMap(row);

      // if that portion of the map is empty, then we're done. otherwise,
      // we need to examine the cached location to verify that it is
      // a match by end key as well.
      if (!matchingRegions.isEmpty()) {
        HRegionLocation possibleRegion = null;
        try {
          possibleRegion = matchingRegions.get(matchingRegions.lastKey());
{code}
is essentially equivalent to 
{code}
possibleRegion = tableLocations.lowerEntry(row).getValue()
{code}

@ronghai: I'm happy to make a patch.

                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "ronghai.ma (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207417#comment-13207417 ] 

ronghai.ma commented on HBASE-5388:
-----------------------------------

Excellent for this patch.
Make another thread for the synchronized and checkReferences.
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, 5388-v4.txt, 5388-v5.txt, 5388-v6.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zhihong Yu updated HBASE-5388:
------------------------------

    Attachment: 5388-v5.txt

Addressed Lars' comments earlier.
Also fixed HConstants.EMPTY_END_ROW in the comment which Lars suggested keeping.

I didn't know Lars was working another patch.
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, 5388-v4.txt, 5388-v5.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zhihong Yu updated HBASE-5388:
------------------------------

    Comment: was deleted

(was: -1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12514303/SoftValueSortedMap.java
  against trunk revision .

    +1 @author.  The patch does not contain any @author tags.

    -1 tests included.  The patch doesn't appear to include any new or modified tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    -1 patch.  The patch command could not apply the patch.

Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/945//console

This message is automatically generated.)
    
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, 5388-v4.txt, 5388-v5.txt, 5388-v6.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207258#comment-13207258 ] 

Zhihong Yu commented on HBASE-5388:
-----------------------------------

lower() isn't in NavigableMap or TreeMap.
And the name is not informative.
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) tunning getCachedLocation method

Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206431#comment-13206431 ] 

Zhihong Yu commented on HBASE-5388:
-----------------------------------

Here you can find source location:
http://hbase.apache.org/source-repository.html
                
> tunning getCachedLocation method
> --------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>              Labels: patch
>
> About 75% improvement。
> 1. Add a method in SoftValueSortedMap
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> 2. Modify getCachedLocation 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206559#comment-13206559 ] 

Zhihong Yu commented on HBASE-5388:
-----------------------------------

I see.

Should this new method, getLower(), be called getLowerValue() ? It returns the value portion of the entry.
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tune HConnectionManager#getCachedLocation method

Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209412#comment-13209412 ] 

Hudson commented on HBASE-5388:
-------------------------------

Integrated in HBase-TRUNK #2663 (See [https://builds.apache.org/job/HBase-TRUNK/2663/])
    HBASE-5388 Tune HConnectionManager#getCachedLocation method (Ronghai Ma) (Revision 1243994)

     Result = FAILURE
tedyu : 
Files : 
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/util/SoftValueSortedMap.java

                
> Tune HConnectionManager#getCachedLocation method
> ------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, 5388-v4.txt, 5388-v5.txt, 5388-v6.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207106#comment-13207106 ] 

Zhihong Yu commented on HBASE-5388:
-----------------------------------

@Lars:
What do you think ?
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HBASE-5388) Tune HConnectionManager#getCachedLocation method

Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zhihong Yu updated HBASE-5388:
------------------------------

    Summary: Tune HConnectionManager#getCachedLocation method  (was: Tuning HConnectionManager#getCachedLocation method)
    
> Tune HConnectionManager#getCachedLocation method
> ------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, 5388-v4.txt, 5388-v5.txt, 5388-v6.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "ronghai.ma (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ronghai.ma updated HBASE-5388:
------------------------------

      Resolution: Fixed
    Hadoop Flags: Reviewed
          Status: Resolved  (was: Patch Available)
    
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, 5388-v4.txt, 5388-v5.txt, 5388-v6.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "ronghai.ma (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206572#comment-13206572 ] 

ronghai.ma commented on HBASE-5388:
-----------------------------------

rename the method SoftValueSortedMap#getLower to SoftValueSortedMap#getLowerValueByKey
{code}
public synchronized V getLowerValueByKey(K key) {
	checkReferences();
	if(!(this.internalMap instanceof java.util.NavigableMap)){
	  SortedMap<K,SoftValue<K,V>> sortedMap = this.internalMap.headMap(key);
	  if(sortedMap==null){
		return null;
	  }
	  SoftValue<K,V> value = sortedMap.get(sortedMap.lastKey());
	  if(value==null){
		return null;
	  }
	  if (value.get() == null) {
	    this.internalMap.remove(key);
	    return null;
	  }
	  return value.get();
	}
	
	Map.Entry<K,SoftValue<K,V>> entry = ((TreeMap<K, SoftValue<K,V>>)this.internalMap).lowerEntry(key);
	if(entry==null){
		return null;
	}
	SoftValue<K,V> value=entry.getValue();
	if(value==null){
		return null;
	} 
	if (value.get() == null) {
      this.internalMap.remove(key);
      return null;
    }
	return value.get();
  }
{code}
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: HConnectionManager.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "ronghai.ma (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207367#comment-13207367 ] 

ronghai.ma commented on HBASE-5388:
-----------------------------------

yes, splited region will be deleted in locateRegionInMeta.
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, 5388-v4.txt, 5388-v5.txt, 5388-v6.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207382#comment-13207382 ] 

Lars Hofhansl commented on HBASE-5388:
--------------------------------------

@ronghai: Null is returned either way, whether we remove it or not.

Anyway, I guess we do the same in get(key), so let's leave it in.

                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, 5388-v4.txt, 5388-v5.txt, 5388-v6.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207363#comment-13207363 ] 

Lars Hofhansl commented on HBASE-5388:
--------------------------------------

@Ted: patch crossing :)

                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, 5388-v4.txt, 5388-v5.txt, 5388-v6.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207309#comment-13207309 ] 

Lars Hofhansl commented on HBASE-5388:
--------------------------------------

I had a look. Having SoftValueSortedMap implement NavigableMap just opens up a new can of worms.
NavigableMap has lowerEntry, but what should that return? A SoftValue? That means the value of the returned entry can become null at any point.

Let's go with the original patch and just add the reference to the concrete SoftValueSortedMap type.
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207101#comment-13207101 ] 

Hadoop QA commented on HBASE-5388:
----------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12514383/5388-v3.txt
  against trunk revision .

    +1 @author.  The patch does not contain any @author tags.

    -1 tests included.  The patch doesn't appear to include any new or modified tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    -1 javadoc.  The javadoc tool appears to have generated -136 warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    -1 findbugs.  The patch appears to introduce 157 new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit warnings.

     -1 core tests.  The patch failed these unit tests:
                       org.apache.hadoop.hbase.io.hfile.TestHFileBlock
                  org.apache.hadoop.hbase.mapreduce.TestImportTsv
                  org.apache.hadoop.hbase.mapred.TestTableMapReduce
                  org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/952//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/952//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/952//console

This message is automatically generated.
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "ronghai.ma (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207355#comment-13207355 ] 

ronghai.ma commented on HBASE-5388:
-----------------------------------

{code}
if (value.get() == null) {
  this.internalMap.remove(key);
  return null;
}
{code}
This is needed for splited regions.
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, 5388-v4.txt, 5388-v5.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "ronghai.ma (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207264#comment-13207264 ] 

ronghai.ma commented on HBASE-5388:
-----------------------------------

As the class is called SoftValueSortedMap, how about SoftValueTreeMap or SoftValueNavigableMap
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206666#comment-13206666 ] 

Hadoop QA commented on HBASE-5388:
----------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12514315/5388-v2.txt
  against trunk revision .

    +1 @author.  The patch does not contain any @author tags.

    -1 tests included.  The patch doesn't appear to include any new or modified tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    -1 javadoc.  The javadoc tool appears to have generated -136 warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    -1 findbugs.  The patch appears to introduce 157 new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit warnings.

     -1 core tests.  The patch failed these unit tests:
                       org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
                  org.apache.hadoop.hbase.mapred.TestTableMapReduce
                  org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks
                  org.apache.hadoop.hbase.mapreduce.TestImportTsv

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/948//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/948//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/948//console

This message is automatically generated.
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206508#comment-13206508 ] 

Lars Hofhansl commented on HBASE-5388:
--------------------------------------

Wow, nice find. Should consider back porting at least to 0.92
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "ronghai.ma (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207373#comment-13207373 ] 

ronghai.ma commented on HBASE-5388:
-----------------------------------

Without remove operation,
are synchronized and checkRefrences() needed here?
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, 5388-v4.txt, 5388-v5.txt, 5388-v6.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Lars Hofhansl (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Lars Hofhansl updated HBASE-5388:
---------------------------------

    Attachment: 5388-v4.txt

How about this one? This has still type-safety everywhere.
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, 5388-v4.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206573#comment-13206573 ] 

Lars Hofhansl commented on HBASE-5388:
--------------------------------------

Usually a diff is better: you can use "svn diff" to do that.
Let's have a more standard lowerEntry rather then getLower.

                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: HConnectionManager.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HBASE-5388) tunning getCachedLocation method

Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zhihong Yu updated HBASE-5388:
------------------------------

     Description: 
About 75% improvement in execution time.
1. Add a method in SoftValueSortedMap
{code}
public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
  return ((TreeMap) this.internalMap).lowerEntry(key);
}
{code}
2. Modify getCachedLocation
{code} 
Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
  if (tEntry != null) {
  HRegionLocation possibleRegion = tEntry.getValue();
  //other code
}
{code}

  was:
About 75% improvement。
1. Add a method in SoftValueSortedMap
public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
  return ((TreeMap) this.internalMap).lowerEntry(key);
}

2. Modify getCachedLocation 
Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
  if (tEntry != null) {
  HRegionLocation possibleRegion = tEntry.getValue();
  //other code
}

    Release Note:   (was: About 75% improvement。
1. Add a method in SoftValueSortedMap
public synchronized <K, V> Entry<K, V> lowerEntry(K key) { return ((TreeMap) this.internalMap).lowerEntry(key); }

2. Modify getCachedLocation 
Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
if (tEntry != null) { HRegionLocation possibleRegion = tEntry.getValue(); //other code })
    
> tunning getCachedLocation method
> --------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>              Labels: patch
>
> About 75% improvement in execution time.
> 1. Add a method in SoftValueSortedMap
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

[jira] [Updated] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "ronghai.ma (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ronghai.ma updated HBASE-5388:
------------------------------

    Attachment: SoftValueSortedMap.java

rename the method's name and change the TreeMap to NavigableMap of the if condition
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HBASE-5388) tunning getCachedLocation method

Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zhihong Yu updated HBASE-5388:
------------------------------

    Fix Version/s: 0.94.0
    
> tunning getCachedLocation method
> --------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>
> About 75% improvement in execution time.
> 1. Add a method in SoftValueSortedMap
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zhihong Yu updated HBASE-5388:
------------------------------

    Description: 
About 75% improvement in execution time.
1. Add the following method in SoftValueSortedMap:
{code}
public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
  return ((TreeMap) this.internalMap).lowerEntry(key);
}
{code}
2. Modify getCachedLocation:
{code} 
Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
  if (tEntry != null) {
  HRegionLocation possibleRegion = tEntry.getValue();
  //other code
}
{code}

  was:
About 75% improvement in execution time.
1. Add a method in SoftValueSortedMap
{code}
public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
  return ((TreeMap) this.internalMap).lowerEntry(key);
}
{code}
2. Modify getCachedLocation
{code} 
Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
  if (tEntry != null) {
  HRegionLocation possibleRegion = tEntry.getValue();
  //other code
}
{code}

        Summary: Tuning HConnectionManager#getCachedLocation method  (was: tunning getCachedLocation method)
    
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) Tuning HConnectionManager#getCachedLocation method

Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207051#comment-13207051 ] 

stack commented on HBASE-5388:
------------------------------

bq. The javadoc involving 'greatest' comes from javadoc for lowerEntry().

Ok

v3 is good by me except for the HCM pollution w/ SoftValueSortedMap.  I looked at making it implement NavigableMap and its 18 extra methods.  They could all throw unsupported as per Lars.  I'm fine w/ this patch though.
                
> Tuning HConnectionManager#getCachedLocation method
> --------------------------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.90.5
>            Reporter: ronghai.ma
>            Assignee: ronghai.ma
>              Labels: patch
>             Fix For: 0.94.0
>
>         Attachments: 5388-v2.txt, 5388-v3.txt, HConnectionManager.java, SoftValueSortedMap.java, SoftValueSortedMap.java
>
>
> About 75% improvement in execution time.
> 1. Add the following method in SoftValueSortedMap:
> {code}
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> {code}
> 2. Modify getCachedLocation:
> {code} 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HBASE-5388) tunning getCachedLocation method

Posted by "ronghai.ma (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HBASE-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206422#comment-13206422 ] 

ronghai.ma commented on HBASE-5388:
-----------------------------------

My pleasure, and two questions:
1. the svn location
2. my account

Yes, measeured in excution time.
                
> tunning getCachedLocation method
> --------------------------------
>
>                 Key: HBASE-5388
>                 URL: https://issues.apache.org/jira/browse/HBASE-5388
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: ronghai.ma
>
> About 75% improvement。
> 1. Add a method in SoftValueSortedMap
> public synchronized <K, V> Entry<K, V> lowerEntry(K key) {
>   return ((TreeMap) this.internalMap).lowerEntry(key);
> }
> 2. Modify getCachedLocation 
> Map.Entry<byte[], HRegionLocation> tEntry = tableLocations.lowerEntry(row);
>   if (tEntry != null) {
>   HRegionLocation possibleRegion = tEntry.getValue();
>   //other code
> }

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira