You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@shardingsphere.apache.org by Juan Pan <pa...@apache.org> on 2020/07/29 10:58:04 UTC

[DISCUSS] Speed metadata loading when ShardingSphere initialize

Hi community,



### Backgroud




ShardingSphere is aiming at simplifying the configuration workload 

and placing itself as `distributed-database` as possible 

to help users manage all the tables (no matter whether it is a sharding table or not).




It is better for users to **use ShardingSphere 

and do not care about** sharding tables, non-sharding tables, encrypted tables, etc. 

But some configurations are necessary. :)




Based on that, we have to load metadata from all the tables. 

But the problem is that this process is time costing when there are thousands of tables to load. 

This discussion is about **speeding the metadata loading**. 




### Solution




After some of the researches, we propose another solution to solve this issue.




Currently, **a rough solution** is that 

we can load metadata through [information_schema](https://en.wikipedia.org/wiki/Information_schema) 

from different databases. (Thanks for your work! @Yonglun)




Specifically, we do some queries from `information_schema` 

 and skip JDBC' call for databases. 

In addition, the JDBC loading method **will remain** for conservative treatment.




### Plan

In the beginning, I will design the basic framework and interfaces 

and then implement `MySQL` loading process. 

After that, we would like to ask our community for help 

to implement `PostgreSQL`, `Oracle`, `SQLServer`, and `H2`. 

Looking forward to your involvement!




Last but not least, **welcome your discussion and thoughts about this issue and solution**!



 Juan Pan (Trista)
                         
Senior DBA & PMC of Apache ShardingSphere
E-mail: panjuan@apache.org