You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "chaijunjie (Jira)" <ji...@apache.org> on 2025/09/04 08:44:00 UTC
[jira] [Comment Edited] (HBASE-29401) HBASE-29401 Support invalidate meta cache when tables dropped or disabled
[ https://issues.apache.org/jira/browse/HBASE-29401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18018058#comment-18018058 ]
chaijunjie edited comment on HBASE-29401 at 9/4/25 8:43 AM:
------------------------------------------------------------
[~zhangduo] Sorry sir, I checked branch-2, the meta-cache in client is different with asyn-client, so the origin code logic only support async-client, dou you think we should also support normal client, it not use AsyncRegionLocator, maybe we could add a chore to check it? just make sure it, then I will backport it to branch-2
thanks
was (Author: JIRAUSER286971):
[~zhangduo] Sorry sir, I checked branch-2, the meta-cache in client is different with asyn-client, so the origin code logic only support async-client, dou you think we should also support normal client, it not use AsyncRegionLocator, maybe we could add a chore to check it? thanks
> HBASE-29401 Support invalidate meta cache when tables dropped or disabled
> -------------------------------------------------------------------------
>
> Key: HBASE-29401
> URL: https://issues.apache.org/jira/browse/HBASE-29401
> Project: HBase
> Issue Type: Improvement
> Components: Client
> Affects Versions: 2.6.1
> Reporter: chaijunjie
> Assignee: chaijunjie
> Priority: Major
> Labels: metaCache, pull-request-available
> Fix For: 3.0.0-beta-2
>
>
> There are one senior:
> we have 2 apps to read/write HBase tables for a long time...
> we could condisder as APP1 and APP2
> APP1 create a new HBase table everyday, and write some data to it, then never write/read it, it is just a daily-table(temporary table)...
> APP2 will drop old table(before 30 days)....
> Then the metaCache in APP1's conn, will contains too many old tables's region locations...even never clear these after the table dropped...only when we create a new hbase conn, will decrease memory used...
> when we drop table using another conn(APP), will cause memleak on other APP's metacache...
> we could not to control to invalidate meta cache when some tables never visit again...
> I add this test to TestMetaCache.java to reproduce it, everyone could use this UT to test, but I am not sure, is this seniro is normaly? or there are some invalidate meta supported? if not let me check if we could limit the meta cache...
> Create one table using conn1, and locate it regions...then drop it by conn2,,,the meta cache of conn1 will keep the table region locations ...
> {code:java}
> @Test
> public void test2Apps() throws Throwable {
> TableName tableName = TableName.valueOf("test2Apps");
> ColumnFamilyDescriptor cf =
> ColumnFamilyDescriptorBuilder.newBuilder(Bytes.toBytes("cf")).build();
> TableDescriptor tbn = TableDescriptorBuilder.newBuilder(tableName).setColumnFamily(cf).build();
> try (ConnectionImplementation conn1 = (ConnectionImplementation) ConnectionFactory.createConnection(
> TEST_UTIL.getConfiguration());
> ConnectionImplementation conn2 = (ConnectionImplementation) ConnectionFactory.createConnection(
> TEST_UTIL.getConfiguration());) {
> try (Admin admin1 = conn1.getAdmin()) {
> admin1.createTable(tbn);
> conn1.getRegionLocator(tableName).getAllRegionLocations();
> Assert.assertEquals(1, conn1.getNumberOfCachedRegionLocations(tableName));
> }
> try (Admin admin2 = conn2.getAdmin()) {
> admin2.disableTable(tableName);
> admin2.deleteTable(tableName);
> conn2.getRegionLocator(tableName).getAllRegionLocations();
> Assert.assertEquals(0, conn2.getNumberOfCachedRegionLocations(tableName));
> }
> Assert.assertEquals(1, conn1.getNumberOfCachedRegionLocations(tableName));
> }
> }
> {code}
> -------------------------------------------------------------
> There 2 soultions:
> 1. use caffeine cache instead of ConcurrentMap in MetaCache, it could help limit memory use, but it will cause performance degradation....
> https://github.com/ben-manes/caffeine/wiki/benchmarks
> 2. use a async thread in conn to invalidate meta cache if table not enbabled...but also could limit the memory if never delete temporary table...
> is there any other soultion?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)