[GitHub] [hbase] saintstack commented on a change in pull request #2352: HBASE-24925 SCP reduce unnecessary get requests

[GitBox <gi...@apache.org>]

saintstack commented on a change in pull request #2352:
URL: https://github.com/apache/hbase/pull/2352#discussion_r485151085



##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
##########
@@ -1556,6 +1556,7 @@ public RegionInfo loadRegionFromMeta(String regionEncodedName) throws UnknownReg
   }
 
   private void loadMeta() throws IOException {
+    master.getTableStateManager().asyncLoadTableState();

Review comment:
       We are doing a full async scan of meta on the table state column followed by another full scan of the meta table?
   Why not just do one table scan that gets both table state and whatever is done below in visitMeta visitor? If the table state is in the first row found when we cross into a new table, and at this time, we save the table state to the TableStateManager cache, won't this have us go to cache on all subsequent table lookups: i.e. less machinery to obtain your objective above?

##########
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
##########
@@ -318,4 +326,39 @@ protected void deleteZooKeeper(TableName tableName) {
       LOG.warn("Failed deleting table state from zookeeper", e);
     }
   }
+
+  public void asyncLoadTableState() {
+    new Thread(() -> {
+      LOG.info("load table state...");
+      ThreadFactoryBuilder builder = new ThreadFactoryBuilder().setDaemon(true).setNameFormat("TableStateFetcher" + "-%1$d");
+      ExecutorService
+        executorService = Executors.newFixedThreadPool(master.getConfiguration().getInt("hbase.table.state.fetcher.threads",10), builder.build());
+
+      try {
+        List<Result> results = MetaTableAccessor.fullScan(master.getConnection(), ClientMetaTableAccessor.QueryType.TABLE);
+        List<Future<Void>> futures = new ArrayList<>();
+        results.forEach(r -> futures.add(executorService.submit(new Callable<Void>() {

Review comment:
       This is a Scan of the Table column in meta? Why we have an executor w/ pool of ten threads to process the result of the Read?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org