You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2021/06/06 12:06:35 UTC
[GitHub] [shardingsphere] tristaZero opened a new issue #10678: Improve the initialization of OptimizeContextFactory
tristaZero opened a new issue #10678:
URL: https://github.com/apache/shardingsphere/issues/10678
### Background
`OptimizeContextFactory` is created in the constructor of `StandardMetaDataContexts` with `metaDataMap`.
If you dive into the constructor of `OptimizeContextFactory` and `MetaDataContextsBuilder`, you will find both handlings of them need loading table metadata. Therefore, this issue is to reduce the unnecessary and duplicated table metadata loading.
### Expected result
- Create `OptimizeContextFactory` in `MetaDataContextsBuilder`
- Share the table metadata loading between `OptimizeContextFactory` and `Map<String, ShardingSphereMetaData>` of `StandardMetaDataContexts`
### How
- The entrance is `buildSchema()` of `MetaDataContextsBuilder`
- Change the result of `SchemaBuilder.build()` to `Map<ActualTableMetadata, LogicTableMetadata>`
In `SchemaBuilder`,
`addRuleConfiguredTables` returns `logicTableMetadata`. On the other hand `appendRemainTables(materials, result);` returns `actualTableMetadata`. So we need to call `load()` and `decorate()` of `TableMetaDataBuilder` respectively to get the acutal and logic ones.
- Assemble `ShardingSphereSchema` using the list of logic table metadata
- Create `OptimizeContextFactory` by the list of actual table metadata
- Remove the deprecated constructors of `FederateSchemaMetadata` and `FederateTableMetadata`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] sizzlecar commented on issue #10678: Improve the initialization of OptimizeContextFactory
Posted by GitBox <gi...@apache.org>.
sizzlecar commented on issue #10678:
URL: https://github.com/apache/shardingsphere/issues/10678#issuecomment-855677292
I can do it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] totalo commented on issue #10678: Improve the initialization of OptimizeContextFactory
Posted by GitBox <gi...@apache.org>.
totalo commented on issue #10678:
URL: https://github.com/apache/shardingsphere/issues/10678#issuecomment-855672224
Hi can I try this ？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] tristaZero commented on issue #10678: Improve the initialization of OptimizeContextFactory
Posted by GitBox <gi...@apache.org>.
tristaZero commented on issue #10678:
URL: https://github.com/apache/shardingsphere/issues/10678#issuecomment-859310443
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] totalo commented on issue #10678: Improve the initialization of OptimizeContextFactory
Posted by GitBox <gi...@apache.org>.
totalo commented on issue #10678:
URL: https://github.com/apache/shardingsphere/issues/10678#issuecomment-859348151
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] LeeGuoPing commented on issue #10678: Improve the initialization of OptimizeContextFactory
Posted by GitBox <gi...@apache.org>.
LeeGuoPing commented on issue #10678:
URL: https://github.com/apache/shardingsphere/issues/10678#issuecomment-855674896
please assign it to me
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] yinyichao commented on issue #10678: Improve the initialization of OptimizeContextFactory
Posted by GitBox <gi...@apache.org>.
yinyichao commented on issue #10678:
URL: https://github.com/apache/shardingsphere/issues/10678#issuecomment-855766606
let me try
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] liulong99 removed a comment on issue #10678: Improve the initialization of OptimizeContextFactory
Posted by GitBox <gi...@apache.org>.
liulong99 removed a comment on issue #10678:
URL: https://github.com/apache/shardingsphere/issues/10678#issuecomment-855776281
please assign it to me
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] liulong99 commented on issue #10678: Improve the initialization of OptimizeContextFactory
Posted by GitBox <gi...@apache.org>.
liulong99 commented on issue #10678:
URL: https://github.com/apache/shardingsphere/issues/10678#issuecomment-855776281
please assign it to me
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] Beyondeclipse commented on issue #10678: Improve the initialization of OptimizeContextFactory
Posted by GitBox <gi...@apache.org>.
Beyondeclipse commented on issue #10678:
URL: https://github.com/apache/shardingsphere/issues/10678#issuecomment-865714539
@tristaZero I found the root cause of slowing down my system is loading all 'TableMetaData' twice:
1. ShardingSphereDataSource -> MetaDataContextsBuilder.build -> MetaDataContextsBuilder.buildMetaData -> SchemaBuilder.build
2. ShardingSphereDataSource -> MetaDataContextsBuilder.build -> StandardMetaDataContexts -> OptimizeContextFactory -> FederateSchemaMetadatas -> FederateSchemaMetadata.initTables
The first way is faster than the second one by using multi-thread and DialectTableMetaDataLoader. It seams that the simple way is to make the loading once and I create a pull request for it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] Beyondeclipse commented on issue #10678: Improve the initialization of OptimizeContextFactory
Posted by GitBox <gi...@apache.org>.
Beyondeclipse commented on issue #10678:
URL: https://github.com/apache/shardingsphere/issues/10678#issuecomment-861476913
Hi @tristaZero, In my opinion, the "ShardingSphereDataSource" should be created and inited faster, and the "StandardMetaDataContexts" should be lazy loaded to avoid slow down the system. Now, "ShardingSphereDataSource" will direct invoke "OptimizeContextFactory" to call "FederateLogicSchema.initTables" which slow down the system with hundreds of databases in startups.
Can this be improved?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] tristaZero closed issue #10678: Improve the initialization of OptimizeContextFactory
Posted by GitBox <gi...@apache.org>.
tristaZero closed issue #10678:
URL: https://github.com/apache/shardingsphere/issues/10678
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org