(doris) branch branch-2.0 updated: [improvement](jdbc catalog)Optimize JDBC Catalog refresh to reduce frequent client creation. (#40261)

[mo...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

morningman pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/branch-2.0 by this push:
     new c46f1d38c1b [improvement](jdbc catalog)Optimize JDBC Catalog refresh to reduce frequent client creation. (#40261)
c46f1d38c1b is described below

commit c46f1d38c1b4be585546dbc7a5d97339102c4ad9
Author: zy-kkk <zh...@gmail.com>
AuthorDate: Mon Sep 2 23:42:16 2024 +0800

    [improvement](jdbc catalog)Optimize JDBC Catalog refresh to reduce frequent client creation. (#40261)
    
    In the previous JDBC Catalog refresh behavior, each refresh would close
    and recreate the JdbcClient. During this process, we observed that some
    JDBC drivers create classloader-level shared threads when the JdbcClient
    is repeatedly instantiated. These threads cannot be garbage collected.
    Since we use the JdbcClient as the context class loader for the JDBC
    driver, frequent creation leads to a buildup of non-recyclable shared
    threads in the JVM. This PR changes the refresh behavior to update the
    cache only, rather than recreating the client. Recreating the client is
    unnecessary when the Catalog configuration has not changed.
---
 .../apache/doris/datasource/jdbc/JdbcExternalCatalog.java    | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/fe/fe-core/src/main/java/org/apache/doris/datasource/jdbc/JdbcExternalCatalog.java b/fe/fe-core/src/main/java/org/apache/doris/datasource/jdbc/JdbcExternalCatalog.java
index 8f9bca79fcc..aabe9d140ee 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/datasource/jdbc/JdbcExternalCatalog.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/datasource/jdbc/JdbcExternalCatalog.java
@@ -101,11 +101,9 @@ public class JdbcExternalCatalog extends ExternalCatalog {
     }
 
     @Override
-    public void onRefresh(boolean invalidCache) {
-        super.onRefresh(invalidCache);
-        if (jdbcClient != null) {
-            jdbcClient.closeClient();
-        }
+    public void notifyPropertiesUpdated(Map<String, String> updatedProps) {
+        super.notifyPropertiesUpdated(updatedProps);
+        this.onClose();
     }
 
     @Override
@@ -113,6 +111,7 @@ public class JdbcExternalCatalog extends ExternalCatalog {
         super.onClose();
         if (jdbcClient != null) {
             jdbcClient.closeClient();
+            jdbcClient = null;
         }
     }
 
@@ -205,6 +204,9 @@ public class JdbcExternalCatalog extends ExternalCatalog {
 
     @Override
     protected void initLocalObjectsImpl() {
+        if (jdbcClient != null) {
+            return;
+        }
         JdbcClientConfig jdbcClientConfig = new JdbcClientConfig()
                 .setCatalog(this.name)
                 .setUser(getJdbcUser())


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org