You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2021/07/09 08:27:52 UTC
[GitHub] [shardingsphere] darkaquarius opened a new issue #11250: the application starts too slowly if there are too many tables
darkaquarius opened a new issue #11250:
URL: https://github.com/apache/shardingsphere/issues/11250
There are 7000+ tables in my database, most of them are single tables.
When the application starts, it will scan all the tables, and each table will scan the table structure, causing my program to take 10 minutes.Can we optimize the startup speed?
ShardingRule:
```
public ShardingRule(final ShardingRuleConfiguration config, final DatabaseType databaseType, final Map<String, DataSource> dataSourceMap) {
......
singleTableRules = SingleTableRuleLoader.load(databaseType, dataSourceMap, getExcludedTables());
......
}
```
FederateSchemaMetadata
```
private void initTables(final ShardingSphereMetaData metaData) throws SQLException {
......
Map<String, Collection<DataNode>> tableDataNodes = getTableDataNodes(dataNodeRules);
......
for (Entry<String, Collection<DataNode>> entry : tableDataNodes.entrySet()) {
......
}
}
```
ShardingRule:
```
@Override
public Map<String, Collection<DataNode>> getAllDataNodes() {
......
singleTableRules.forEach((key, value) -> result.put(key, Collections.singleton(new DataNode(value.getDataSourceName(), value.getTableName()))));
......
}
```
TableMetaDataLoader
```
private static boolean isTableExist(final Connection connection, final String tableNamePattern) throws SQLException {
try (ResultSet resultSet = connection.getMetaData().getTables(connection.getCatalog(), connection.getSchema(), tableNamePattern, null)) {
return resultSet.next();
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] strongduanmu commented on issue #11250: the application starts too slowly if there are too many tables
Posted by GitBox <gi...@apache.org>.
strongduanmu commented on issue #11250:
URL: https://github.com/apache/shardingsphere/issues/11250#issuecomment-927763149
Metadata loading performance has been optimized, please use the master branch to test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] tristaZero commented on issue #11250: the application starts too slowly if there are too many tables
Posted by GitBox <gi...@apache.org>.
tristaZero commented on issue #11250:
URL: https://github.com/apache/shardingsphere/issues/11250#issuecomment-877132687
This refers to the metadata loading improvement for single tables.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] strongduanmu commented on issue #11250: the application starts too slowly if there are too many tables
Posted by GitBox <gi...@apache.org>.
strongduanmu commented on issue #11250:
URL: https://github.com/apache/shardingsphere/issues/11250#issuecomment-879556414
@darkaquarius Can you try the latest master branch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] strongduanmu closed issue #11250: the application starts too slowly if there are too many tables
Posted by GitBox <gi...@apache.org>.
strongduanmu closed issue #11250:
URL: https://github.com/apache/shardingsphere/issues/11250
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] misskeyjoy commented on issue #11250: the application starts too slowly if there are too many tables
Posted by GitBox <gi...@apache.org>.
misskeyjoy commented on issue #11250:
URL: https://github.com/apache/shardingsphere/issues/11250#issuecomment-918163357
Use below code, configure one additional data source
// 配置真实数据源
// 这里有个bug 配置 一个 DataSource 的时候,启动很慢,
Map<String, DataSource> dataSourceMap = ImmutableMap.of("ds0", hikariDataSource,"ds1",hikariDataSource);
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] darkaquarius commented on issue #11250: the application starts too slowly if there are too many tables
Posted by GitBox <gi...@apache.org>.
darkaquarius commented on issue #11250:
URL: https://github.com/apache/shardingsphere/issues/11250#issuecomment-877014743
@tristaZero
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] strongduanmu commented on issue #11250: the application starts too slowly if there are too many tables
Posted by GitBox <gi...@apache.org>.
strongduanmu commented on issue #11250:
URL: https://github.com/apache/shardingsphere/issues/11250#issuecomment-918702624
Hi @misskeyjoy , we have performed some performance optimizations on the master branch. Can you use the master branch to test whether this problem exists?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org