[GitHub] [metron] Apache9 commented on a change in pull request #1483: METRON-2217 Migrate current HBase client from HTableInterface to Table

[GitBox <gi...@apache.org>]

Apache9 commented on a change in pull request #1483: METRON-2217 Migrate current HBase client from HTableInterface to Table
URL: https://github.com/apache/metron/pull/1483#discussion_r315474945
 
 

 ##########
 File path: metron-platform/metron-hbase/metron-hbase-common/src/main/java/org/apache/metron/hbase/HTableProvider.java
 ##########
 @@ -18,13 +18,49 @@
 package org.apache.metron.hbase;
 
 import java.io.IOException;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
 import org.apache.hadoop.conf.Configuration;
-import org.apache.hadoop.hbase.client.HTable;
-import org.apache.hadoop.hbase.client.HTableInterface;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Connection;
+import org.apache.hadoop.hbase.client.ConnectionFactory;
+import org.apache.hadoop.hbase.client.Table;
 
 public class HTableProvider implements TableProvider {
-    @Override
-    public HTableInterface getTable(Configuration config, String tableName) throws IOException {
-        return new HTable(config, tableName);
+
+  private static class RetryingConnection {
+
+    private Configuration config;
+    private Connection conn;
+
+    RetryingConnection(Configuration config) {
+      this.config = config;
+    }
+
+    public Connection getUnderlying() throws IOException {
+      if (conn == null || conn.isClosed()) {
+        conn = ConnectionFactory.createConnection(config);
+      }
+      return conn;
     }
+  }
+
+  /**
+   * We have to handle serialization issues with Storm via indirections. Rather than re-implement
+   * the interface everywhere we touch HBase, we can use a lazy initialization scheme to encapsulate
+   * this within the HTableProvider. This is a sort of poor man's connection pool.
+   */
+  private static Map<Configuration, ThreadLocal<RetryingConnection>> connMap = new ConcurrentHashMap<>();
 
 Review comment:
   And actually, Connection is thread safe, so you only need to cache Tables. And since the Table is getting from Connection, which means it will not make new TCP connections to HBaseCluster, maybe we even do not need to cache the Tables, as it is only some in memory operations.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services