You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by yi...@apache.org on 2024/02/22 12:35:26 UTC
(doris) branch branch-2.1 updated (f65876d8030 -> 18f9ca92420)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a change to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
from f65876d8030 [Feature](explode) support explode map type (#30151)
new 7fd53a92b89 fix partial-update doc (#30612)
new f42935560de [tools] remove useless conf (#31295)
new 18f9ca92420 [fix](jdbc catalog) Fix Resource Closing Logic After Connection Abort (#31219)
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../data-operate/update-delete/partial-update.md | 2 +-
.../data-operate/update-delete/partial-update.md | 2 +-
.../java/org/apache/doris/jdbc/JdbcExecutor.java | 60 ++++++++++++----------
tools/tpcds-tools/conf/opt/opt_sf1.sql | 1 -
tools/tpcds-tools/conf/opt/opt_sf100.sql | 1 -
tools/tpcds-tools/conf/opt/opt_sf1000.sql | 1 -
tools/tpcds-tools/conf/opt/opt_sf10000.sql | 1 -
7 files changed, 36 insertions(+), 32 deletions(-)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
(doris) 02/03: [tools] remove useless conf (#31295)
Posted by yi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
commit f42935560de7508dc404adc9cb00b5ae61741fa8
Author: xzj7019 <13...@users.noreply.github.com>
AuthorDate: Thu Feb 22 20:29:54 2024 +0800
[tools] remove useless conf (#31295)
Co-authored-by: zhongjian.xzj <zh...@zhongjianxzjdeMacBook-Pro.local>
---
tools/tpcds-tools/conf/opt/opt_sf1.sql | 1 -
tools/tpcds-tools/conf/opt/opt_sf100.sql | 1 -
tools/tpcds-tools/conf/opt/opt_sf1000.sql | 1 -
tools/tpcds-tools/conf/opt/opt_sf10000.sql | 1 -
4 files changed, 4 deletions(-)
diff --git a/tools/tpcds-tools/conf/opt/opt_sf1.sql b/tools/tpcds-tools/conf/opt/opt_sf1.sql
index dafe9f37134..ea43723c30d 100644
--- a/tools/tpcds-tools/conf/opt/opt_sf1.sql
+++ b/tools/tpcds-tools/conf/opt/opt_sf1.sql
@@ -1,2 +1 @@
set global runtime_filter_wait_time_ms=1000;
-set global enable_nereids_rules='ELIMINATE_GROUP_BY_KEY';
diff --git a/tools/tpcds-tools/conf/opt/opt_sf100.sql b/tools/tpcds-tools/conf/opt/opt_sf100.sql
index dafe9f37134..ea43723c30d 100644
--- a/tools/tpcds-tools/conf/opt/opt_sf100.sql
+++ b/tools/tpcds-tools/conf/opt/opt_sf100.sql
@@ -1,2 +1 @@
set global runtime_filter_wait_time_ms=1000;
-set global enable_nereids_rules='ELIMINATE_GROUP_BY_KEY';
diff --git a/tools/tpcds-tools/conf/opt/opt_sf1000.sql b/tools/tpcds-tools/conf/opt/opt_sf1000.sql
index 2e42f2ed9e7..17d8fa190f3 100644
--- a/tools/tpcds-tools/conf/opt/opt_sf1000.sql
+++ b/tools/tpcds-tools/conf/opt/opt_sf1000.sql
@@ -1,2 +1 @@
set global runtime_filter_wait_time_ms=10000;
-set global enable_nereids_rules='ELIMINATE_GROUP_BY_KEY';
diff --git a/tools/tpcds-tools/conf/opt/opt_sf10000.sql b/tools/tpcds-tools/conf/opt/opt_sf10000.sql
index 47233ff6910..ef11bad93de 100644
--- a/tools/tpcds-tools/conf/opt/opt_sf10000.sql
+++ b/tools/tpcds-tools/conf/opt/opt_sf10000.sql
@@ -1,2 +1 @@
set global runtime_filter_wait_time_ms=100000;
-set global enable_nereids_rules='ELIMINATE_GROUP_BY_KEY';
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
(doris) 01/03: fix partial-update doc (#30612)
Posted by yi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
commit 7fd53a92b89e6b2a5edeefbfc6e4e685d97957ad
Author: bingquanzhao <bi...@icloud.com>
AuthorDate: Thu Feb 22 20:10:40 2024 +0800
fix partial-update doc (#30612)
---
docs/en/docs/data-operate/update-delete/partial-update.md | 2 +-
docs/zh-CN/docs/data-operate/update-delete/partial-update.md | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/docs/en/docs/data-operate/update-delete/partial-update.md b/docs/en/docs/data-operate/update-delete/partial-update.md
index cd40886ffee..021d44c7d06 100644
--- a/docs/en/docs/data-operate/update-delete/partial-update.md
+++ b/docs/en/docs/data-operate/update-delete/partial-update.md
@@ -124,7 +124,7 @@ If you are using the Flink Connector, you need to add the following configuratio
'sink.properties.partial_columns' = 'true',
```
-Also, specify the columns to be loaded in `sink.properties.column` (it must include all key columns, or else updates cannot be performed).
+Also, specify the columns to be loaded in `sink.properties.columns` (it must include all key columns, or else updates cannot be performed).
#### INSERT INTO
diff --git a/docs/zh-CN/docs/data-operate/update-delete/partial-update.md b/docs/zh-CN/docs/data-operate/update-delete/partial-update.md
index ce1bcb8b737..95310fe8a5e 100644
--- a/docs/zh-CN/docs/data-operate/update-delete/partial-update.md
+++ b/docs/zh-CN/docs/data-operate/update-delete/partial-update.md
@@ -121,7 +121,7 @@ partial_columns:true
```
'sink.properties.partial_columns' = 'true',
```
-同时在`sink.properties.column`中指定要导入的列（必须包含所有key列，不然无法更新）
+同时在`sink.properties.columns`中指定要导入的列（必须包含所有key列，不然无法更新）
#### INSERT INTO
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
(doris) 03/03: [fix](jdbc catalog) Fix Resource Closing Logic After Connection Abort (#31219)
Posted by yi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
commit 18f9ca92420cf6b61c9972f732912bf8f55569b7
Author: zy-kkk <zh...@gmail.com>
AuthorDate: Thu Feb 22 20:34:09 2024 +0800
[fix](jdbc catalog) Fix Resource Closing Logic After Connection Abort (#31219)
* [improvement](jdbc catalog) Optimize the closing logic of Jdbc connection after abort
* [improvement](jdbc catalog) Optimize the closing logic of Jdbc connection after abort
* fix
---
.../java/org/apache/doris/jdbc/JdbcExecutor.java | 60 ++++++++++++----------
1 file changed, 34 insertions(+), 26 deletions(-)
diff --git a/fe/be-java-extensions/jdbc-scanner/src/main/java/org/apache/doris/jdbc/JdbcExecutor.java b/fe/be-java-extensions/jdbc-scanner/src/main/java/org/apache/doris/jdbc/JdbcExecutor.java
index 8c684219af3..3f00643d126 100644
--- a/fe/be-java-extensions/jdbc-scanner/src/main/java/org/apache/doris/jdbc/JdbcExecutor.java
+++ b/fe/be-java-extensions/jdbc-scanner/src/main/java/org/apache/doris/jdbc/JdbcExecutor.java
@@ -134,32 +134,20 @@ public class JdbcExecutor {
LOG.error("Error cancelling statement", e);
}
}
- if (conn != null && resultSet != null) {
- abortReadConnection(conn, resultSet, tableType);
- try {
- resultSet.close();
- } catch (SQLException e) {
- LOG.error("Error closing resultSet", e);
- }
- try {
- stmt.close();
- } catch (SQLException e) {
- LOG.error("Error closing statement", e);
- }
- }
- } finally {
- if (conn != null && !conn.isClosed()) {
- try {
- conn.close();
- } catch (SQLException e) {
- LOG.error("Error closing connection", e);
- }
+
+ boolean shouldAbort = conn != null && resultSet != null
+ && (tableType == TOdbcTableType.MYSQL || tableType == TOdbcTableType.SQLSERVER);
+ boolean aborted = false; // Used to record whether the abort operation is performed
+ if (shouldAbort) {
+ aborted = abortReadConnection(conn, resultSet, tableType);
}
- }
- if (config.getConnectionPoolMinSize() == 0) {
- // Close and remove the datasource if necessary
- if (druidDataSource != null) {
+ // If no abort operation is performed, the resource needs to be closed manually
+ if (!aborted) {
+ closeResources(resultSet, stmt, conn);
+ }
+ } finally {
+ if (config.getConnectionPoolMinSize() == 0 && druidDataSource != null) {
druidDataSource.close();
JdbcDataSource.getDataSource().getSourcesMap().remove(config.createCacheKey());
druidDataSource = null;
@@ -167,13 +155,33 @@ public class JdbcExecutor {
}
}
- public void abortReadConnection(Connection connection, ResultSet resultSet, TOdbcTableType tableType)
+ private void closeResources(AutoCloseable... closeables) {
+ for (AutoCloseable closeable : closeables) {
+ if (closeable != null) {
+ try {
+ if (closeable instanceof Connection) {
+ if (!((Connection) closeable).isClosed()) {
+ closeable.close();
+ }
+ } else {
+ closeable.close();
+ }
+ } catch (Exception e) {
+ LOG.error("Cannot close resource: ", e);
+ }
+ }
+ }
+ }
+
+ public boolean abortReadConnection(Connection connection, ResultSet resultSet, TOdbcTableType tableType)
throws SQLException {
if (!resultSet.isAfterLast() && (tableType == TOdbcTableType.MYSQL || tableType == TOdbcTableType.SQLSERVER)) {
- // Abort connection before closing. Without this, the MySQL driver
+ // Abort connection before closing. Without this, the MySQL/SQLServer driver
// attempts to drain the connection by reading all the results.
connection.abort(MoreExecutors.directExecutor());
+ return true;
}
+ return false;
}
public int read() throws UdfRuntimeException {
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org