You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "whhe (via GitHub)" <gi...@apache.org> on 2023/07/17 06:47:15 UTC

[GitHub] [seatunnel] whhe opened a new pull request, #5096: [Feature][Connector-v2][Jdbc] Refactor AbstractJdbcCatalog

whhe opened a new pull request, #5096:
URL: https://github.com/apache/seatunnel/pull/5096

   <!--
   
   Thank you for contributing to SeaTunnel! Please make sure that your code changes
   are covered with tests. And in case of new features or big changes
   remember to adjust the documentation.
   
   Feel free to ping committers for the review!
   
   ## Contribution Checklist
   
     - Make sure that the pull request corresponds to a [GITHUB issue](https://github.com/apache/seatunnel/issues).
   
     - Name the pull request in the form "[Feature] [component] Title of the pull request", where *Feature* can be replaced by `Hotfix`, `Bug`, etc.
   
     - Minor fixes should be named following this pattern: `[hotfix] [docs] Fix typo in README.md doc`.
   
   -->
   
   ## Purpose of this pull request
   
   <!-- Describe the purpose of this pull request. For example: This pull request adds checkstyle plugin.-->
   
   Extract the common logic of MySqlCatalog and SqlServerCatalog to AbstractJdbcCatalog.
   
   
   ## Check list
   
   * [ ] Code changed are covered with tests, or it does not need tests for reason:
   * [ ] If any new Jar binary package adding in your PR, please add License Notice according
     [New License Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/contribution/new-license.md)
   * [ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
   * [ ] If you are contributing the connector code, please check that the following files are updated:
     1. Update change log that in connector document. For more details you can refer to [connector-v2](https://github.com/apache/seatunnel/tree/dev/docs/en/connector-v2)
     2. Update [plugin-mapping.properties](https://github.com/apache/seatunnel/blob/dev/plugin-mapping.properties) and add new connector information in it
     3. Update the pom file of [seatunnel-dist](https://github.com/apache/seatunnel/blob/dev/seatunnel-dist/pom.xml)
   * [ ] Update the [`release-note`](https://github.com/apache/seatunnel/blob/dev/release-note.md).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] whhe commented on pull request #5096: [Improve][Connector-v2][Jdbc] Refactor AbstractJdbcCatalog

Posted by "whhe (via GitHub)" <gi...@apache.org>.
whhe commented on PR #5096:
URL: https://github.com/apache/seatunnel/pull/5096#issuecomment-1702218590

   All tests passed, PTAL @ic4y @liugddx @EricJoy2048  


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] XiaoJiang521 commented on a diff in pull request #5096: [Improve][Connector-v2][Jdbc] Refactor AbstractJdbcCatalog

Posted by "XiaoJiang521 (via GitHub)" <gi...@apache.org>.
XiaoJiang521 commented on code in PR #5096:
URL: https://github.com/apache/seatunnel/pull/5096#discussion_r1286652443


##########
seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/catalog/sqlserver/SqlServerCatalog.java:
##########
@@ -292,99 +184,27 @@ private SeaTunnelDataType<?> fromJdbcType(String typeName, int precision, int sc
     }
 
     @Override
-    protected boolean createTableInternal(TablePath tablePath, CatalogTable table)
-            throws CatalogException {
-
-        String createTableSql =
-                SqlServerCreateTableSqlBuilder.builder(tablePath, table).build(tablePath, table);
-        log.info("create table sql: {}", createTableSql);
-        try (Connection conn = DriverManager.getConnection(defaultUrl, username, pwd);
-                PreparedStatement ps = conn.prepareStatement(createTableSql)) {
-            System.out.println(createTableSql);
-            return ps.execute();
-        } catch (Exception e) {
-            throw new CatalogException(
-                    String.format("Failed creating table %s", tablePath.getFullName()), e);
-        }
+    protected String getCreateTableSql(TablePath tablePath, CatalogTable table) {
+        return SqlServerCreateTableSqlBuilder.builder(tablePath, table).build(tablePath, table);
     }
 
     @Override
-    protected boolean dropTableInternal(TablePath tablePath) throws CatalogException {
-        String dbUrl = getUrlFromDatabaseName(tablePath.getDatabaseName());
-        try (Connection conn = DriverManager.getConnection(dbUrl, username, pwd);
-                PreparedStatement ps =
-                        conn.prepareStatement(
-                                String.format(
-                                        "DROP TABLE IF EXISTS %s", tablePath.getFullName()))) {
-            // Will there exist concurrent drop for one table?
-            return ps.execute();
-        } catch (SQLException e) {
-            throw new CatalogException(
-                    String.format("Failed dropping table %s", tablePath.getFullName()), e);
-        }
+    protected String getDropTableSql(TablePath tablePath) {
+        return String.format("DROP TABLE IF EXISTS %s", tablePath.getFullName());
     }
 
     @Override
-    protected boolean createDatabaseInternal(String databaseName) throws CatalogException {
-        try (Connection conn = DriverManager.getConnection(defaultUrl, username, pwd);
-                PreparedStatement ps =
-                        conn.prepareStatement(
-                                String.format("CREATE DATABASE `%s`", databaseName))) {
-            return ps.execute();
-        } catch (Exception e) {
-            throw new CatalogException(
-                    String.format(
-                            "Failed creating database %s in catalog %s",
-                            databaseName, this.catalogName),
-                    e);
-        }
+    protected String getCreateDatabaseSql(String databaseName) {
+        return String.format("CREATE DATABASE `%s`", databaseName);
     }
 
     @Override
-    protected boolean dropDatabaseInternal(String databaseName) throws CatalogException {
-        try (Connection conn = DriverManager.getConnection(defaultUrl, username, pwd);
-                PreparedStatement ps =
-                        conn.prepareStatement(
-                                String.format("DROP DATABASE IF EXISTS `%s`;", databaseName))) {
-            return ps.execute();
-        } catch (Exception e) {
-            throw new CatalogException(
-                    String.format(
-                            "Failed dropping database %s in catalog %s",
-                            databaseName, this.catalogName),
-                    e);
-        }
+    protected String getDropDatabaseSql(String databaseName) {
+        return String.format("DROP DATABASE IF EXISTS `%s`;", databaseName);

Review Comment:
   sqlserver is [%s]. no ``



##########
seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/catalog/sqlserver/SqlServerCatalog.java:
##########
@@ -292,99 +184,27 @@ private SeaTunnelDataType<?> fromJdbcType(String typeName, int precision, int sc
     }
 
     @Override
-    protected boolean createTableInternal(TablePath tablePath, CatalogTable table)
-            throws CatalogException {
-
-        String createTableSql =
-                SqlServerCreateTableSqlBuilder.builder(tablePath, table).build(tablePath, table);
-        log.info("create table sql: {}", createTableSql);
-        try (Connection conn = DriverManager.getConnection(defaultUrl, username, pwd);
-                PreparedStatement ps = conn.prepareStatement(createTableSql)) {
-            System.out.println(createTableSql);
-            return ps.execute();
-        } catch (Exception e) {
-            throw new CatalogException(
-                    String.format("Failed creating table %s", tablePath.getFullName()), e);
-        }
+    protected String getCreateTableSql(TablePath tablePath, CatalogTable table) {
+        return SqlServerCreateTableSqlBuilder.builder(tablePath, table).build(tablePath, table);
     }
 
     @Override
-    protected boolean dropTableInternal(TablePath tablePath) throws CatalogException {
-        String dbUrl = getUrlFromDatabaseName(tablePath.getDatabaseName());
-        try (Connection conn = DriverManager.getConnection(dbUrl, username, pwd);
-                PreparedStatement ps =
-                        conn.prepareStatement(
-                                String.format(
-                                        "DROP TABLE IF EXISTS %s", tablePath.getFullName()))) {
-            // Will there exist concurrent drop for one table?
-            return ps.execute();
-        } catch (SQLException e) {
-            throw new CatalogException(
-                    String.format("Failed dropping table %s", tablePath.getFullName()), e);
-        }
+    protected String getDropTableSql(TablePath tablePath) {
+        return String.format("DROP TABLE IF EXISTS %s", tablePath.getFullName());
     }
 
     @Override
-    protected boolean createDatabaseInternal(String databaseName) throws CatalogException {
-        try (Connection conn = DriverManager.getConnection(defaultUrl, username, pwd);
-                PreparedStatement ps =
-                        conn.prepareStatement(
-                                String.format("CREATE DATABASE `%s`", databaseName))) {
-            return ps.execute();
-        } catch (Exception e) {
-            throw new CatalogException(
-                    String.format(
-                            "Failed creating database %s in catalog %s",
-                            databaseName, this.catalogName),
-                    e);
-        }
+    protected String getCreateDatabaseSql(String databaseName) {
+        return String.format("CREATE DATABASE `%s`", databaseName);

Review Comment:
   sqlserver is CREATE DATABASE [%s]. no ``



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] XiaoJiang521 commented on a diff in pull request #5096: [Improve][Connector-v2][Jdbc] Refactor AbstractJdbcCatalog

Posted by "XiaoJiang521 (via GitHub)" <gi...@apache.org>.
XiaoJiang521 commented on code in PR #5096:
URL: https://github.com/apache/seatunnel/pull/5096#discussion_r1286702558


##########
seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/catalog/sqlserver/SqlServerCatalog.java:
##########
@@ -292,99 +184,27 @@ private SeaTunnelDataType<?> fromJdbcType(String typeName, int precision, int sc
     }
 
     @Override
-    protected boolean createTableInternal(TablePath tablePath, CatalogTable table)
-            throws CatalogException {
-
-        String createTableSql =
-                SqlServerCreateTableSqlBuilder.builder(tablePath, table).build(tablePath, table);
-        log.info("create table sql: {}", createTableSql);
-        try (Connection conn = DriverManager.getConnection(defaultUrl, username, pwd);
-                PreparedStatement ps = conn.prepareStatement(createTableSql)) {
-            System.out.println(createTableSql);
-            return ps.execute();
-        } catch (Exception e) {
-            throw new CatalogException(
-                    String.format("Failed creating table %s", tablePath.getFullName()), e);
-        }
+    protected String getCreateTableSql(TablePath tablePath, CatalogTable table) {
+        return SqlServerCreateTableSqlBuilder.builder(tablePath, table).build(tablePath, table);
     }
 
     @Override
-    protected boolean dropTableInternal(TablePath tablePath) throws CatalogException {
-        String dbUrl = getUrlFromDatabaseName(tablePath.getDatabaseName());
-        try (Connection conn = DriverManager.getConnection(dbUrl, username, pwd);
-                PreparedStatement ps =
-                        conn.prepareStatement(
-                                String.format(
-                                        "DROP TABLE IF EXISTS %s", tablePath.getFullName()))) {
-            // Will there exist concurrent drop for one table?
-            return ps.execute();
-        } catch (SQLException e) {
-            throw new CatalogException(
-                    String.format("Failed dropping table %s", tablePath.getFullName()), e);
-        }
+    protected String getDropTableSql(TablePath tablePath) {
+        return String.format("DROP TABLE IF EXISTS %s", tablePath.getFullName());
     }
 
     @Override
-    protected boolean createDatabaseInternal(String databaseName) throws CatalogException {
-        try (Connection conn = DriverManager.getConnection(defaultUrl, username, pwd);
-                PreparedStatement ps =
-                        conn.prepareStatement(
-                                String.format("CREATE DATABASE `%s`", databaseName))) {
-            return ps.execute();
-        } catch (Exception e) {
-            throw new CatalogException(
-                    String.format(
-                            "Failed creating database %s in catalog %s",
-                            databaseName, this.catalogName),
-                    e);
-        }
+    protected String getCreateDatabaseSql(String databaseName) {
+        return String.format("CREATE DATABASE `%s`", databaseName);

Review Comment:
   
   Yes, this is a historical bug. Thank you for your effort.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] EricJoy2048 closed pull request #5096: [Improve][Connector-v2][Jdbc] Refactor AbstractJdbcCatalog

Posted by "EricJoy2048 (via GitHub)" <gi...@apache.org>.
EricJoy2048 closed pull request #5096: [Improve][Connector-v2][Jdbc] Refactor AbstractJdbcCatalog
URL: https://github.com/apache/seatunnel/pull/5096


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] whhe commented on pull request #5096: [Improve][Connector-v2][Jdbc] Refactor AbstractJdbcCatalog

Posted by "whhe (via GitHub)" <gi...@apache.org>.
whhe commented on PR #5096:
URL: https://github.com/apache/seatunnel/pull/5096#issuecomment-1695975569

   I add `testCatalog` method in `AbstractJdbcIT` and enable it for MySql, Oracle and SqlServer in this pr, looks like these IT cases finally passed.
   
   As PG does not use the same base IT class, I want to modify it in another pr, what do you think? @ic4y 
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] whhe commented on pull request #5096: [Improve][Connector-v2][Jdbc] Refactor AbstractJdbcCatalog

Posted by "whhe (via GitHub)" <gi...@apache.org>.
whhe commented on PR #5096:
URL: https://github.com/apache/seatunnel/pull/5096#issuecomment-1669495800

   @XiaoJiang521 Comments addressed, what do you think for now.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] EricJoy2048 commented on pull request #5096: [Improve][Connector-v2][Jdbc] Refactor AbstractJdbcCatalog

Posted by "EricJoy2048 (via GitHub)" <gi...@apache.org>.
EricJoy2048 commented on PR #5096:
URL: https://github.com/apache/seatunnel/pull/5096#issuecomment-1643423899

   > > > Mockito
   > > 
   > > 
   > > Refer to `seatunnel-e2e/connector-jdbc-e2e`
   > 
   > So I should add IT cases in module connector-jdbc-e2e instead of UT cases in the modified module, am I right?
   
   Yes, you can add IT for this pr.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] EricJoy2048 merged pull request #5096: [Improve][Connector-v2][Jdbc] Refactor AbstractJdbcCatalog

Posted by "EricJoy2048 (via GitHub)" <gi...@apache.org>.
EricJoy2048 merged PR #5096:
URL: https://github.com/apache/seatunnel/pull/5096


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] XiaoJiang521 commented on pull request #5096: [Improve][Connector-v2][Jdbc] Refactor AbstractJdbcCatalog

Posted by "XiaoJiang521 (via GitHub)" <gi...@apache.org>.
XiaoJiang521 commented on PR #5096:
URL: https://github.com/apache/seatunnel/pull/5096#issuecomment-1705992180

   LGTM @ic4y 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] XiaoJiang521 commented on a diff in pull request #5096: [Improve][Connector-v2][Jdbc] Refactor AbstractJdbcCatalog

Posted by "XiaoJiang521 (via GitHub)" <gi...@apache.org>.
XiaoJiang521 commented on code in PR #5096:
URL: https://github.com/apache/seatunnel/pull/5096#discussion_r1286624758


##########
seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/catalog/AbstractJdbcCatalog.java:
##########
@@ -88,58 +98,107 @@ public AbstractJdbcCatalog(
         this.defaultUrl = urlInfo.getOrigin();
         this.suffix = urlInfo.getSuffix();
         this.defaultSchema = Optional.ofNullable(defaultSchema);
+        this.connectionMap = new ConcurrentHashMap<>();
     }
 
+    protected abstract String getCatalogType();
+
     @Override
     public String getDefaultDatabase() {
         return defaultDatabase;
     }
 
-    public String getCatalogName() {
-        return catalogName;
+    protected Connection getConnection(String url) {
+        if (connectionMap.containsKey(url)) {
+            return connectionMap.get(url);
+        }
+        try {
+            Connection connection = DriverManager.getConnection(url, username, pwd);
+            connectionMap.put(url, connection);
+            return connection;
+        } catch (SQLException e) {
+            throw new CatalogException(String.format("Failed connecting to %s via JDBC.", url), e);
+        }
     }
 
-    public String getUsername() {
-        return username;
+    @Override
+    public void open() throws CatalogException {
+        getConnection(defaultUrl);
+        LOG.info("Catalog {} established connection to {}", catalogName, defaultUrl);
     }
 
-    public String getPassword() {
-        return pwd;
+    @Override
+    public void close() throws CatalogException {
+        for (Map.Entry<String, Connection> entry : connectionMap.entrySet()) {
+            try {
+                entry.getValue().close();
+            } catch (SQLException e) {
+                throw new CatalogException(
+                        String.format("Failed to close %s via JDBC.", entry.getKey()), e);
+            }
+        }
+        connectionMap.clear();
+        LOG.info("Catalog {} closing", catalogName);
     }

Review Comment:
   
   <img width="979" alt="image" src="https://github.com/apache/seatunnel/assets/131635688/9d0324cf-d0df-4f69-9b86-5bfe2d7d0d19">
   Wouldn't it be problematic to abstract this part? For example, closing the catalog in MySQL, but the catalogs in Oracle and PostgreSQL are still in use. Wouldn't closing like this cause the connections in Oracle and PostgreSQL to be closed as well? @whhe 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] XiaoJiang521 commented on a diff in pull request #5096: [Improve][Connector-v2][Jdbc] Refactor AbstractJdbcCatalog

Posted by "XiaoJiang521 (via GitHub)" <gi...@apache.org>.
XiaoJiang521 commented on code in PR #5096:
URL: https://github.com/apache/seatunnel/pull/5096#discussion_r1286624758


##########
seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/catalog/AbstractJdbcCatalog.java:
##########
@@ -88,58 +98,107 @@ public AbstractJdbcCatalog(
         this.defaultUrl = urlInfo.getOrigin();
         this.suffix = urlInfo.getSuffix();
         this.defaultSchema = Optional.ofNullable(defaultSchema);
+        this.connectionMap = new ConcurrentHashMap<>();
     }
 
+    protected abstract String getCatalogType();
+
     @Override
     public String getDefaultDatabase() {
         return defaultDatabase;
     }
 
-    public String getCatalogName() {
-        return catalogName;
+    protected Connection getConnection(String url) {
+        if (connectionMap.containsKey(url)) {
+            return connectionMap.get(url);
+        }
+        try {
+            Connection connection = DriverManager.getConnection(url, username, pwd);
+            connectionMap.put(url, connection);
+            return connection;
+        } catch (SQLException e) {
+            throw new CatalogException(String.format("Failed connecting to %s via JDBC.", url), e);
+        }
     }
 
-    public String getUsername() {
-        return username;
+    @Override
+    public void open() throws CatalogException {
+        getConnection(defaultUrl);
+        LOG.info("Catalog {} established connection to {}", catalogName, defaultUrl);
     }
 
-    public String getPassword() {
-        return pwd;
+    @Override
+    public void close() throws CatalogException {
+        for (Map.Entry<String, Connection> entry : connectionMap.entrySet()) {
+            try {
+                entry.getValue().close();
+            } catch (SQLException e) {
+                throw new CatalogException(
+                        String.format("Failed to close %s via JDBC.", entry.getKey()), e);
+            }
+        }
+        connectionMap.clear();
+        LOG.info("Catalog {} closing", catalogName);
     }

Review Comment:
   
   <img width="979" alt="image" src="https://github.com/apache/seatunnel/assets/131635688/9d0324cf-d0df-4f69-9b86-5bfe2d7d0d19">
   Wouldn't it be problematic to abstract this part? For example, closing the catalog in MySQL, but the catalogs in Oracle and PostgreSQL are still in use. Wouldn't closing like this cause the connections in Oracle and PostgreSQL to be closed as well? @whhe 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] whhe commented on a diff in pull request #5096: [Improve][Connector-v2][Jdbc] Refactor AbstractJdbcCatalog

Posted by "whhe (via GitHub)" <gi...@apache.org>.
whhe commented on code in PR #5096:
URL: https://github.com/apache/seatunnel/pull/5096#discussion_r1286673270


##########
seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/catalog/sqlserver/SqlServerCatalog.java:
##########
@@ -292,99 +184,27 @@ private SeaTunnelDataType<?> fromJdbcType(String typeName, int precision, int sc
     }
 
     @Override
-    protected boolean createTableInternal(TablePath tablePath, CatalogTable table)
-            throws CatalogException {
-
-        String createTableSql =
-                SqlServerCreateTableSqlBuilder.builder(tablePath, table).build(tablePath, table);
-        log.info("create table sql: {}", createTableSql);
-        try (Connection conn = DriverManager.getConnection(defaultUrl, username, pwd);
-                PreparedStatement ps = conn.prepareStatement(createTableSql)) {
-            System.out.println(createTableSql);
-            return ps.execute();
-        } catch (Exception e) {
-            throw new CatalogException(
-                    String.format("Failed creating table %s", tablePath.getFullName()), e);
-        }
+    protected String getCreateTableSql(TablePath tablePath, CatalogTable table) {
+        return SqlServerCreateTableSqlBuilder.builder(tablePath, table).build(tablePath, table);
     }
 
     @Override
-    protected boolean dropTableInternal(TablePath tablePath) throws CatalogException {
-        String dbUrl = getUrlFromDatabaseName(tablePath.getDatabaseName());
-        try (Connection conn = DriverManager.getConnection(dbUrl, username, pwd);
-                PreparedStatement ps =
-                        conn.prepareStatement(
-                                String.format(
-                                        "DROP TABLE IF EXISTS %s", tablePath.getFullName()))) {
-            // Will there exist concurrent drop for one table?
-            return ps.execute();
-        } catch (SQLException e) {
-            throw new CatalogException(
-                    String.format("Failed dropping table %s", tablePath.getFullName()), e);
-        }
+    protected String getDropTableSql(TablePath tablePath) {
+        return String.format("DROP TABLE IF EXISTS %s", tablePath.getFullName());
     }
 
     @Override
-    protected boolean createDatabaseInternal(String databaseName) throws CatalogException {
-        try (Connection conn = DriverManager.getConnection(defaultUrl, username, pwd);
-                PreparedStatement ps =
-                        conn.prepareStatement(
-                                String.format("CREATE DATABASE `%s`", databaseName))) {
-            return ps.execute();
-        } catch (Exception e) {
-            throw new CatalogException(
-                    String.format(
-                            "Failed creating database %s in catalog %s",
-                            databaseName, this.catalogName),
-                    e);
-        }
+    protected String getCreateDatabaseSql(String databaseName) {
+        return String.format("CREATE DATABASE `%s`", databaseName);

Review Comment:
   I just copied the sql templates from https://github.com/apache/seatunnel/blob/dev/seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/catalog/sqlserver/SqlServerCatalog.java
   
   Will fix it later.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] whhe commented on pull request #5096: [Improve][Connector-v2][Jdbc] Refactor AbstractJdbcCatalog

Posted by "whhe (via GitHub)" <gi...@apache.org>.
whhe commented on PR #5096:
URL: https://github.com/apache/seatunnel/pull/5096#issuecomment-1697352986

   Thanks to help from @EricJoy2048, CI jobs are totally green now.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] liugddx commented on pull request #5096: [Improve][Connector-v2][Jdbc] Refactor AbstractJdbcCatalog

Posted by "liugddx (via GitHub)" <gi...@apache.org>.
liugddx commented on PR #5096:
URL: https://github.com/apache/seatunnel/pull/5096#issuecomment-1702194590

   > @liugddx Could you please help to retry the failed ut case? Seems the `testJobLevelCheckpointTimeOut` is not completely fixed.
   
   Yes, I found the problem, it's not very stable. I will fix this problem in other PR #5414 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] whhe commented on pull request #5096: [Improve][Connector-v2][Jdbc] Refactor AbstractJdbcCatalog

Posted by "whhe (via GitHub)" <gi...@apache.org>.
whhe commented on PR #5096:
URL: https://github.com/apache/seatunnel/pull/5096#issuecomment-1698790731

   All catalog implementations related to `AbstractJdbcCatalog` are covered by IT cases now, and UT failure is not related.
   
   PTAL  @ic4y @liugddx @EricJoy2048


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] whhe commented on pull request #5096: [Improve][Connector-v2][Jdbc] Refactor AbstractJdbcCatalog

Posted by "whhe (via GitHub)" <gi...@apache.org>.
whhe commented on PR #5096:
URL: https://github.com/apache/seatunnel/pull/5096#issuecomment-1669222458

   The failed case `JdbcPhoenixIT` seems not related to this pr.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] whhe commented on pull request #5096: [Improve][Connector-v2][Jdbc] Refactor AbstractJdbcCatalog

Posted by "whhe (via GitHub)" <gi...@apache.org>.
whhe commented on PR #5096:
URL: https://github.com/apache/seatunnel/pull/5096#issuecomment-1670585888

   @ic4y @EricJoy2048 PTAL


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] TyrantLucifer commented on pull request #5096: [Improve][Connector-v2][Jdbc] Refactor AbstractJdbcCatalog

Posted by "TyrantLucifer (via GitHub)" <gi...@apache.org>.
TyrantLucifer commented on PR #5096:
URL: https://github.com/apache/seatunnel/pull/5096#issuecomment-1637929462

   Please add unit test cases. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] ic4y commented on pull request #5096: [Improve][Connector-v2][Jdbc] Refactor AbstractJdbcCatalog

Posted by "ic4y (via GitHub)" <gi...@apache.org>.
ic4y commented on PR #5096:
URL: https://github.com/apache/seatunnel/pull/5096#issuecomment-1670617012

   At present, no tests cover these modifications, and the UT tests are also not working. We should add a seatunnel-catalog-e2e module under the seatunnel-e2e module and use testcontainers to cover these tests.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] whhe commented on pull request #5096: [Improve][Connector-v2][Jdbc] Refactor AbstractJdbcCatalog

Posted by "whhe (via GitHub)" <gi...@apache.org>.
whhe commented on PR #5096:
URL: https://github.com/apache/seatunnel/pull/5096#issuecomment-1701006622

   @liugddx Could you please help to retry the failed ut case? Seems the `testJobLevelCheckpointTimeOut` is not completely fixed.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] ic4y commented on pull request #5096: [Improve][Connector-v2][Jdbc] Refactor AbstractJdbcCatalog

Posted by "ic4y (via GitHub)" <gi...@apache.org>.
ic4y commented on PR #5096:
URL: https://github.com/apache/seatunnel/pull/5096#issuecomment-1666428779

   @whhe Great commit! Now we just need to resolve the conflicts.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] whhe commented on pull request #5096: [Improve][Connector-v2][Jdbc] Refactor AbstractJdbcCatalog

Posted by "whhe (via GitHub)" <gi...@apache.org>.
whhe commented on PR #5096:
URL: https://github.com/apache/seatunnel/pull/5096#issuecomment-1641895425

   > > Mockito
   > 
   > Refer to `seatunnel-e2e/connector-jdbc-e2e`
   
   So I should add IT cases in module connector-jdbc-e2e instead of UT cases in the modified module, am I right?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] whhe commented on pull request #5096: [Improve][Connector-v2][Jdbc] Refactor AbstractJdbcCatalog

Posted by "whhe (via GitHub)" <gi...@apache.org>.
whhe commented on PR #5096:
URL: https://github.com/apache/seatunnel/pull/5096#issuecomment-1639270197

   > Please add unit test cases.
   
   Seems SeaTunnel does not depend on Mockito now, I want to introduce it to mock jdbc connection, is it OK ? And if so what version should I choose?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] liugddx commented on pull request #5096: [Improve][Connector-v2][Jdbc] Refactor AbstractJdbcCatalog

Posted by "liugddx (via GitHub)" <gi...@apache.org>.
liugddx commented on PR #5096:
URL: https://github.com/apache/seatunnel/pull/5096#issuecomment-1641877022

   > Mockito
   
   Refer to `seatunnel-e2e/connector-jdbc-e2e` 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] whhe commented on pull request #5096: [Improve][Connector-v2][Jdbc] Refactor AbstractJdbcCatalog

Posted by "whhe (via GitHub)" <gi...@apache.org>.
whhe commented on PR #5096:
URL: https://github.com/apache/seatunnel/pull/5096#issuecomment-1667663575

   I rebased the code and seems it's coverd by the IT cases for now. @ic4y @EricJoy2048 @liugddx 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] ic4y commented on pull request #5096: [Improve][Connector-v2][Jdbc] Refactor AbstractJdbcCatalog

Posted by "ic4y (via GitHub)" <gi...@apache.org>.
ic4y commented on PR #5096:
URL: https://github.com/apache/seatunnel/pull/5096#issuecomment-1668844509

   @XiaoJiang521 PTAL


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] XiaoJiang521 commented on pull request #5096: [Improve][Connector-v2][Jdbc] Refactor AbstractJdbcCatalog

Posted by "XiaoJiang521 (via GitHub)" <gi...@apache.org>.
XiaoJiang521 commented on PR #5096:
URL: https://github.com/apache/seatunnel/pull/5096#issuecomment-1670551477

   LGTM


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] whhe commented on pull request #5096: [Improve][Connector-v2][Jdbc] Refactor AbstractJdbcCatalog

Posted by "whhe (via GitHub)" <gi...@apache.org>.
whhe commented on PR #5096:
URL: https://github.com/apache/seatunnel/pull/5096#issuecomment-1670642207

   > At present, no tests cover these modifications, and the UT tests are also not working. We should add a seatunnel-catalog-e2e module under the seatunnel-e2e module and use testcontainers to cover these tests.
   
   Finally got caught with this work again ... will try it later.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] liugddx commented on pull request #5096: [Improve][Connector-v2][Jdbc] Refactor AbstractJdbcCatalog

Posted by "liugddx (via GitHub)" <gi...@apache.org>.
liugddx commented on PR #5096:
URL: https://github.com/apache/seatunnel/pull/5096#issuecomment-1698820292

   > All catalog implementations related to `AbstractJdbcCatalog` are covered by IT cases now, and UT failure is not related.
   > 
   > PTAL @ic4y @liugddx @EricJoy2048
   
   Waiting for #5403 to be merged


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] ic4y commented on pull request #5096: [Improve][Connector-v2][Jdbc] Refactor AbstractJdbcCatalog

Posted by "ic4y (via GitHub)" <gi...@apache.org>.
ic4y commented on PR #5096:
URL: https://github.com/apache/seatunnel/pull/5096#issuecomment-1705910849

   @XiaoJiang521 is familiar with this area, could you please review it?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org