You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "honghua zhu (Created) (JIRA)" <ji...@apache.org> on 2012/02/29 17:21:57 UTC
[jira] [Created] (HBASE-5492) Caching StartKeys and EndKeys of
Regions
Caching StartKeys and EndKeys of Regions
----------------------------------------
Key: HBASE-5492
URL: https://issues.apache.org/jira/browse/HBASE-5492
Project: HBase
Issue Type: Improvement
Components: client
Affects Versions: 0.92.0
Environment: all
Reporter: honghua zhu
Fix For: 0.92.1
Attachments: HBASE-5492.patch
Each call for HTable.getStartEndKeys will read meta table.
In particular,
in the case of client side multi-threaded concurrency statistics,
we must call HTable.coprocessorExec== > getStartKeysInRange ==> getStartEndKeys,
resulting in the need to always scan the meta table.
This is not necessary,
we can implement the HConnectionManager.HConnectionImplementation.locateRegions(byte[] tableName) method,
then, get the StartKeys and EndKeys from the cachedRegionLocations of HConnectionImplementation.
Combined with https://issues.apache.org/jira/browse/HBASE-5491, can improve the performance of statistical
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5492) Caching StartKeys and EndKeys of
Regions
Posted by "honghua zhu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13219716#comment-13219716 ]
honghua zhu commented on HBASE-5492:
------------------------------------
@stack
Using cachedRegionLocations of HConnectionImplementation.
Your question can be converted into who is responsible for refreshing of cachedRegionLocations?
> Caching StartKeys and EndKeys of Regions
> ----------------------------------------
>
> Key: HBASE-5492
> URL: https://issues.apache.org/jira/browse/HBASE-5492
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.92.0
> Environment: all
> Reporter: honghua zhu
> Fix For: 0.92.1
>
> Attachments: HBASE-5492.patch
>
>
> Each call for HTable.getStartEndKeys will read meta table.
> In particular,
> in the case of client side multi-threaded concurrency statistics,
> we must call HTable.coprocessorExec== > getStartKeysInRange ==> getStartEndKeys,
> resulting in the need to always scan the meta table.
> This is not necessary,
> we can implement the HConnectionManager.HConnectionImplementation.locateRegions(byte[] tableName) method,
> then, get the StartKeys and EndKeys from the cachedRegionLocations of HConnectionImplementation.
> Combined with https://issues.apache.org/jira/browse/HBASE-5491, can improve the performance of statistical
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5492) Caching StartKeys and EndKeys of
Regions
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-5492:
-------------------------
Fix Version/s: (was: 0.92.2)
0.92.3
> Caching StartKeys and EndKeys of Regions
> ----------------------------------------
>
> Key: HBASE-5492
> URL: https://issues.apache.org/jira/browse/HBASE-5492
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.92.0
> Environment: all
> Reporter: honghua zhu
> Fix For: 0.92.3
>
> Attachments: HBASE-5492.patch
>
>
> Each call for HTable.getStartEndKeys will read meta table.
> In particular,
> in the case of client side multi-threaded concurrency statistics,
> we must call HTable.coprocessorExec== > getStartKeysInRange ==> getStartEndKeys,
> resulting in the need to always scan the meta table.
> This is not necessary,
> we can implement the HConnectionManager.HConnectionImplementation.locateRegions(byte[] tableName) method,
> then, get the StartKeys and EndKeys from the cachedRegionLocations of HConnectionImplementation.
> Combined with https://issues.apache.org/jira/browse/HBASE-5491, can improve the performance of statistical
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5492) Caching StartKeys and EndKeys of
Regions
Posted by "honghua zhu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
honghua zhu updated HBASE-5492:
-------------------------------
Status: Patch Available (was: Open)
> Caching StartKeys and EndKeys of Regions
> ----------------------------------------
>
> Key: HBASE-5492
> URL: https://issues.apache.org/jira/browse/HBASE-5492
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.92.0
> Environment: all
> Reporter: honghua zhu
> Fix For: 0.92.1
>
> Attachments: HBASE-5492.patch
>
>
> Each call for HTable.getStartEndKeys will read meta table.
> In particular,
> in the case of client side multi-threaded concurrency statistics,
> we must call HTable.coprocessorExec== > getStartKeysInRange ==> getStartEndKeys,
> resulting in the need to always scan the meta table.
> This is not necessary,
> we can implement the HConnectionManager.HConnectionImplementation.locateRegions(byte[] tableName) method,
> then, get the StartKeys and EndKeys from the cachedRegionLocations of HConnectionImplementation.
> Combined with https://issues.apache.org/jira/browse/HBASE-5491, can improve the performance of statistical
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5492) Caching StartKeys and EndKeys of
Regions
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13219345#comment-13219345 ]
Hadoop QA commented on HBASE-5492:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12516578/HBASE-5492.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 javadoc. The javadoc tool appears to have generated -131 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 154 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.wal.TestHLog
org.apache.hadoop.hbase.TestRegionRebalancing
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1061//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1061//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1061//console
This message is automatically generated.
> Caching StartKeys and EndKeys of Regions
> ----------------------------------------
>
> Key: HBASE-5492
> URL: https://issues.apache.org/jira/browse/HBASE-5492
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.92.0
> Environment: all
> Reporter: honghua zhu
> Fix For: 0.92.1
>
> Attachments: HBASE-5492.patch
>
>
> Each call for HTable.getStartEndKeys will read meta table.
> In particular,
> in the case of client side multi-threaded concurrency statistics,
> we must call HTable.coprocessorExec== > getStartKeysInRange ==> getStartEndKeys,
> resulting in the need to always scan the meta table.
> This is not necessary,
> we can implement the HConnectionManager.HConnectionImplementation.locateRegions(byte[] tableName) method,
> then, get the StartKeys and EndKeys from the cachedRegionLocations of HConnectionImplementation.
> Combined with https://issues.apache.org/jira/browse/HBASE-5491, can improve the performance of statistical
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5492) Caching StartKeys and EndKeys of
Regions
Posted by "honghua zhu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
honghua zhu updated HBASE-5492:
-------------------------------
Attachment: HBASE-5492.patch
> Caching StartKeys and EndKeys of Regions
> ----------------------------------------
>
> Key: HBASE-5492
> URL: https://issues.apache.org/jira/browse/HBASE-5492
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.92.0
> Environment: all
> Reporter: honghua zhu
> Fix For: 0.92.1
>
> Attachments: HBASE-5492.patch
>
>
> Each call for HTable.getStartEndKeys will read meta table.
> In particular,
> in the case of client side multi-threaded concurrency statistics,
> we must call HTable.coprocessorExec== > getStartKeysInRange ==> getStartEndKeys,
> resulting in the need to always scan the meta table.
> This is not necessary,
> we can implement the HConnectionManager.HConnectionImplementation.locateRegions(byte[] tableName) method,
> then, get the StartKeys and EndKeys from the cachedRegionLocations of HConnectionImplementation.
> Combined with https://issues.apache.org/jira/browse/HBASE-5491, can improve the performance of statistical
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5492) Caching StartKeys and EndKeys of
Regions
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13220370#comment-13220370 ]
stack commented on HBASE-5492:
------------------------------
bq. Your question can be converted into who is responsible for refreshing of cachedRegionLocations?
You can do that. Its done as the HTable runs but there are no guarantees the cache is complete at any one time, right?
Change + if (tableLocations.size() == 0) { to tableLocations.isEmpty().. the former can be costly compared.
So, IIUC, you are asking to pull all region locations local in locateRegions?
Does this get all table regions? Or just the configured next ten regions?
+ prefetchRegionCache(tableName, null);
> Caching StartKeys and EndKeys of Regions
> ----------------------------------------
>
> Key: HBASE-5492
> URL: https://issues.apache.org/jira/browse/HBASE-5492
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.92.0
> Environment: all
> Reporter: honghua zhu
> Fix For: 0.92.1
>
> Attachments: HBASE-5492.patch
>
>
> Each call for HTable.getStartEndKeys will read meta table.
> In particular,
> in the case of client side multi-threaded concurrency statistics,
> we must call HTable.coprocessorExec== > getStartKeysInRange ==> getStartEndKeys,
> resulting in the need to always scan the meta table.
> This is not necessary,
> we can implement the HConnectionManager.HConnectionImplementation.locateRegions(byte[] tableName) method,
> then, get the StartKeys and EndKeys from the cachedRegionLocations of HConnectionImplementation.
> Combined with https://issues.apache.org/jira/browse/HBASE-5491, can improve the performance of statistical
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5492) Caching StartKeys and EndKeys of
Regions
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13219333#comment-13219333 ]
stack commented on HBASE-5492:
------------------------------
Once filled, how does the cache of table locations get refreshed if a table region splits?
> Caching StartKeys and EndKeys of Regions
> ----------------------------------------
>
> Key: HBASE-5492
> URL: https://issues.apache.org/jira/browse/HBASE-5492
> Project: HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.92.0
> Environment: all
> Reporter: honghua zhu
> Fix For: 0.92.1
>
> Attachments: HBASE-5492.patch
>
>
> Each call for HTable.getStartEndKeys will read meta table.
> In particular,
> in the case of client side multi-threaded concurrency statistics,
> we must call HTable.coprocessorExec== > getStartKeysInRange ==> getStartEndKeys,
> resulting in the need to always scan the meta table.
> This is not necessary,
> we can implement the HConnectionManager.HConnectionImplementation.locateRegions(byte[] tableName) method,
> then, get the StartKeys and EndKeys from the cachedRegionLocations of HConnectionImplementation.
> Combined with https://issues.apache.org/jira/browse/HBASE-5491, can improve the performance of statistical
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira