You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@seata.apache.org by "georliu (via GitHub)" <gi...@apache.org> on 2025/04/18 07:58:32 UTC
[I] 多次使用new DataSourceProxy(dataSource).getConnection()导致oom的问题 [incubator-seata]
georliu opened a new issue, #7303:
URL: https://github.com/apache/incubator-seata/issues/7303
我在项目中使用connection = new DataSourceProxy(dataSource).getConnection()获取数据源，当并并发量起来之后使用监控发现线程数量一直没降下来，最终导致java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached，具体的异常如下
Caused by: java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
at java.base/java.lang.Thread.start0(Native Method)
at java.base/java.lang.Thread.start(Thread.java:809)
at java.base/java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:945)
at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1353)
at io.seata.rm.datasource.sql.struct.TableMetaCacheFactory$TableMetaRefreshHolder.<init>(TableMetaCacheFactory.java:116)
at io.seata.rm.datasource.sql.struct.TableMetaCacheFactory.registerTableMeta(TableMetaCacheFactory.java:87)
at io.seata.rm.datasource.DataSourceProxy.init(DataSourceProxy.java:101)
at io.seata.rm.datasource.DataSourceProxy.<init>(DataSourceProxy.java:82)
at io.seata.rm.datasource.DataSourceProxy.<init>(DataSourceProxy.java:67)
at com.dtt.tax.lowcode.config.apijson.LowCodeSQLExecutor.getConnection(LowCodeSQLExecutor.java:167)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org
For additional commands, e-mail: notifications-help@seata.apache.org
Re: [I] 多次使用new DataSourceProxy(dataSource).getConnection()导致oom的问题 [incubator-seata]
Posted by "georliu (via GitHub)" <gi...@apache.org>.
georliu commented on issue #7303:
URL: https://github.com/apache/incubator-seata/issues/7303#issuecomment-2822912731
> > 我们使用的是APIJSON进行的开发，本地扩展了AbstractSQLExecutor，代码段如下 `// 适配连接池，如果这里能拿到连接池的有效 Connection，则 SQLConfig 不需要配置 dbVersion, dbUri, dbAccount, dbPassword [@OverRide](https://github.com/OverRide) public Connection getConnection(SQLConfig config) throws Exception {
> > ```
> > if (StringUtils.isNotBlank(config.getDBUri())) {
> > ```
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > // Connection c = CACHED_CONNECTION.getIfPresent(config.getDBUri()); Connection c = connectionMap.get(config.getDBUri()); if (c == null || c.isClosed()) { DataSource dataSource; if (DATA_SOURCE_CACHE.containsKey(config.getDBUri())) { dataSource = DATA_SOURCE_CACHE.get(config.getDBUri()); } else { dataSource = createdDataSource(config.isMySQL() ? "com.mysql.cj.jdbc.Driver" : "dm.jdbc.driver.DmDriver", config.getDBUri(), config.getDBAccount(), config.getDBPassword(), "apijson"); DATA_SOURCE_CACHE.put(config.getDBUri(), dataSource); } if (GET == config.getMethod() || HEAD == config.getMethod()) { connection = (Connection) ((DruidDataSource) dataSource).getPooledConnection(); } else { connection = new DataSourceProxy(dataSource).getConnection(); } connectionMap.put(config.getDBUri(), connection); // //CACHED_CONNECTION.put(config.getDBUri(), connection); } return connection; } Connection c = connectionMap.get(config.getDatabase()); if (c == null || c.isClosed()) { try { DynamicRou
tingDataSource ds = (DynamicRoutingDataSource) LowCodeBusinessApplication.getApplicationContext().getBean(DataSource.class); connectionMap.put(config.getDatasource() + "-" + config.getDatabase(), ds.getConnection()); } catch (Exception e) { Log.e(TAG, "getConnection try { " + "DataSource ds = DemoApplication.getApplicationContext().getBean(DataSource.class); .." + "} catch (Exception e) = " + e.getMessage()); } }
> > ```
> > // 必须最后执行 super 方法，因为里面还有事务相关处理。
> > // 如果这里是 return c，则会导致 增删改 多个对象时只有第一个会 commit，即只有第一个对象成功插入数据库表
> > return super.getConnection(config);
> > }`
> > ```
>
> 那就说明你代码有bug呀，把datasourceproxy缓存起来，不要每次去new
好的谢谢
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org
For additional commands, e-mail: notifications-help@seata.apache.org
Re: [I] 多次使用new DataSourceProxy(dataSource).getConnection()导致oom的问题 [incubator-seata]
Posted by "funky-eyes (via GitHub)" <gi...@apache.org>.
funky-eyes commented on issue #7303:
URL: https://github.com/apache/incubator-seata/issues/7303#issuecomment-2819822462
> 我们使用的是APIJSON进行的开发，本地扩展了AbstractSQLExecutor，代码段如下 `// 适配连接池，如果这里能拿到连接池的有效 Connection，则 SQLConfig 不需要配置 dbVersion, dbUri, dbAccount, dbPassword [@OverRide](https://github.com/OverRide) public Connection getConnection(SQLConfig config) throws Exception {
>
> ```
> if (StringUtils.isNotBlank(config.getDBUri())) {
> ```
>
> // Connection c = CACHED_CONNECTION.getIfPresent(config.getDBUri()); Connection c = connectionMap.get(config.getDBUri()); if (c == null || c.isClosed()) { DataSource dataSource; if (DATA_SOURCE_CACHE.containsKey(config.getDBUri())) { dataSource = DATA_SOURCE_CACHE.get(config.getDBUri()); } else { dataSource = createdDataSource(config.isMySQL() ? "com.mysql.cj.jdbc.Driver" : "dm.jdbc.driver.DmDriver", config.getDBUri(), config.getDBAccount(), config.getDBPassword(), "apijson"); DATA_SOURCE_CACHE.put(config.getDBUri(), dataSource); } if (GET == config.getMethod() || HEAD == config.getMethod()) { connection = (Connection) ((DruidDataSource) dataSource).getPooledConnection(); } else { connection = new DataSourceProxy(dataSource).getConnection(); } connectionMap.put(config.getDBUri(), connection); // //CACHED_CONNECTION.put(config.getDBUri(), connection); } return connection; } Connection c = connectionMap.get(config.getDatabase()); if (c == null || c.isClosed()) { try { DynamicRouti
ngDataSource ds = (DynamicRoutingDataSource) LowCodeBusinessApplication.getApplicationContext().getBean(DataSource.class); connectionMap.put(config.getDatasource() + "-" + config.getDatabase(), ds.getConnection()); } catch (Exception e) { Log.e(TAG, "getConnection try { " + "DataSource ds = DemoApplication.getApplicationContext().getBean(DataSource.class); .." + "} catch (Exception e) = " + e.getMessage()); } }
>
> ```
> // 必须最后执行 super 方法，因为里面还有事务相关处理。
> // 如果这里是 return c，则会导致 增删改 多个对象时只有第一个会 commit，即只有第一个对象成功插入数据库表
> return super.getConnection(config);
> }`
> ```
那就说明你代码有bug呀，把datasourceproxy缓存起来，不要每次去new
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org
For additional commands, e-mail: notifications-help@seata.apache.org
Re: [I] 多次使用new DataSourceProxy(dataSource).getConnection()导致oom的问题 [incubator-seata]
Posted by "georliu (via GitHub)" <gi...@apache.org>.
georliu commented on issue #7303:
URL: https://github.com/apache/incubator-seata/issues/7303#issuecomment-2815160091
完整的日志如下，目前看起来是在每次执行APIJSON的executeSQL方法时，获取connectionMap中未关闭的连接，如果已关闭就会通过connection = new DataSourceProxy(dataSource).getConnection()重新建立连接
Caused by: java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
at java.base/java.lang.Thread.start0(Native Method)
at java.base/java.lang.Thread.start(Thread.java:809)
at java.base/java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:945)
at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1353)
at io.seata.rm.datasource.sql.struct.TableMetaCacheFactory$TableMetaRefreshHolder.<init>(TableMetaCacheFactory.java:116)
at io.seata.rm.datasource.sql.struct.TableMetaCacheFactory.registerTableMeta(TableMetaCacheFactory.java:87)
at io.seata.rm.datasource.DataSourceProxy.init(DataSourceProxy.java:101)
at io.seata.rm.datasource.DataSourceProxy.<init>(DataSourceProxy.java:82)
at io.seata.rm.datasource.DataSourceProxy.<init>(DataSourceProxy.java:67)
at com.dtt.tax.lowcode.config.apijson.LowCodeSQLExecutor.getConnection(LowCodeSQLExecutor.java:167)
at apijson.orm.AbstractSQLExecutor.getStatement(AbstractSQLExecutor.java:1154)
at apijson.orm.AbstractSQLExecutor.getStatement(AbstractSQLExecutor.java:1123)
at apijson.orm.AbstractSQLExecutor.executeUpdate(AbstractSQLExecutor.java:1412)
at apijson.orm.SQLExecutor.executeUpdate(SQLExecutor.java:80)
at apijson.orm.AbstractSQLExecutor.execute(AbstractSQLExecutor.java:242)
at apijson.orm.AbstractParser.executeSQL(AbstractParser.java:2032)
at com.dtt.tax.lowcode.config.apijson.LowCodeParser.executeSQL(LowCodeParser.java:173)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org
For additional commands, e-mail: notifications-help@seata.apache.org
Re: [I] 多次使用new DataSourceProxy(dataSource).getConnection()导致oom的问题 [incubator-seata]
Posted by "georliu (via GitHub)" <gi...@apache.org>.
georliu commented on issue #7303:
URL: https://github.com/apache/incubator-seata/issues/7303#issuecomment-2814867263
我在之前使用1.5.2的版本未出现过该问题，升级到1.7.0之后就出现了，如果能提供一下排查思路的话十分感谢
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org
For additional commands, e-mail: notifications-help@seata.apache.org
Re: [I] 多次使用new DataSourceProxy(dataSource).getConnection()导致oom的问题 [incubator-seata]
Posted by "georliu (via GitHub)" <gi...@apache.org>.
georliu closed issue #7303: 多次使用new DataSourceProxy(dataSource).getConnection()导致oom的问题
URL: https://github.com/apache/incubator-seata/issues/7303
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org
For additional commands, e-mail: notifications-help@seata.apache.org
Re: [I] 多次使用new DataSourceProxy(dataSource).getConnection()导致oom的问题 [incubator-seata]
Posted by "funky-eyes (via GitHub)" <gi...@apache.org>.
funky-eyes commented on issue #7303:
URL: https://github.com/apache/incubator-seata/issues/7303#issuecomment-2814933699
为什么要不断的new DataSourceProxy(dataSource)去获取链接？你会close这个datasource吗？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org
For additional commands, e-mail: notifications-help@seata.apache.org
Re: [I] 多次使用new DataSourceProxy(dataSource).getConnection()导致oom的问题 [incubator-seata]
Posted by "georliu (via GitHub)" <gi...@apache.org>.
georliu commented on issue #7303:
URL: https://github.com/apache/incubator-seata/issues/7303#issuecomment-2815127005
我们使用的是APIJSON进行的开发，本地扩展了AbstractSQLExecutor，代码段如下
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org
For additional commands, e-mail: notifications-help@seata.apache.org