You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2020/05/22 09:28:04 UTC
[GitHub] [shardingsphere] cws1981 opened a new issue #5755: sharding-jdbc slow start
cws1981 opened a new issue #5755:
URL: https://github.com/apache/shardingsphere/issues/5755
**sharding-jdbc 4.0**
sharding-jdbc load all table metaData when start ,I have more then 600 table so spend too much time when start,can I assign table to load ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] tristaZero commented on issue #5755: sharding-jdbc slow start
Posted by GitBox <gi...@apache.org>.
tristaZero commented on issue #5755:
URL: https://github.com/apache/shardingsphere/issues/5755#issuecomment-698200050
Hi @cws1981 ,
I am sorry to give my comment late. The good news that the performance of starting ShardingProxy has been greatly improved already! Thanks for @xbkaishui, who took much effort in #6212.
Presently, welcome your any test for the `master branch` (Package it local firstly). Your feedback will be the best gift for the community.
Best,
Trista
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] yu199195 commented on issue #5755: sharding-jdbc slow start
Posted by GitBox <gi...@apache.org>.
yu199195 commented on issue #5755:
URL: https://github.com/apache/shardingsphere/issues/5755#issuecomment-632759486
you can config this in you yaml.
```yml
spring.shardingsphere.check.table.metadata.enabled =false
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] cws1981 commented on issue #5755: sharding-jdbc slow start
Posted by GitBox <gi...@apache.org>.
cws1981 commented on issue #5755:
URL: https://github.com/apache/shardingsphere/issues/5755#issuecomment-632996629
我想最好是 把要分表的表单独放一个scheme 减少表的数量
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] yu199195 commented on issue #5755: sharding-jdbc slow start
Posted by GitBox <gi...@apache.org>.
yu199195 commented on issue #5755:
URL: https://github.com/apache/shardingsphere/issues/5755#issuecomment-632724440
you can not check metadata。
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] liuqiankun93 commented on issue #5755: sharding-jdbc slow start
Posted by GitBox <gi...@apache.org>.
liuqiankun93 commented on issue #5755:
URL: https://github.com/apache/shardingsphere/issues/5755#issuecomment-634516596
> this option can avoid sharding tables metadata loading:
>
> > spring.shardingsphere.check.table.metadata.enabled =false
my version is 4.1.0
I use jdbc config.The code is
`Properties properties = new Properties();
properties.setProperty("check.table.metadata.enabled","false");
DataSource dataSource = ShardingDataSourceFactory.createDataSource(dataSourceMap, shardingRuleConfig,properties);`
and the log is
`16:36:08.105 [main] INFO org.apache.shardingsphere.core.log.ConfigurationLogger - Properties:
check.table.metadata.enabled: 'false'
16:36:08.115 [main] INFO ShardingSphere-metadata - Loading 1 logic tables' meta data.
16:36:08.809 [main] INFO ShardingSphere-metadata - Loading 240 tables' meta data.`
the config It doesn't seem to be working
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] yu199195 commented on issue #5755: sharding-jdbc slow start
Posted by GitBox <gi...@apache.org>.
yu199195 commented on issue #5755:
URL: https://github.com/apache/shardingsphere/issues/5755#issuecomment-632985170
@liuqiankun93
now，we use executerService load table metadata，
you can config this：
```
spring.shardingsphere.max.connections.size.per.query =1 (default 1, changed by you)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] cws1981 commented on issue #5755: sharding-jdbc slow start
Posted by GitBox <gi...@apache.org>.
cws1981 commented on issue #5755:
URL: https://github.com/apache/shardingsphere/issues/5755#issuecomment-633788265
It works, now it is two thirds less than before
`max.connections.size.per.query: '100'`
`20/05/26 11:03:36 INFO RMI TCP Connection(2)-127.0.0.1 ShardingSphere-metadata: Loading 2 logic tables' meta data.
20/05/26 11:03:36 INFO RMI TCP Connection(2)-127.0.0.1 ShardingSphere-metadata: Loading 647 tables' meta data.
20/05/26 11:04:02 INFO RMI TCP Connection(2)-127.0.0.1 ShardingSphere-metadata: Meta data load finished, cost 26610 milliseconds.`
The local test used to take more than 70 seconds, but now only 27 seconds
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] yu199195 edited a comment on issue #5755: sharding-jdbc slow start
Posted by GitBox <gi...@apache.org>.
yu199195 edited a comment on issue #5755:
URL: https://github.com/apache/shardingsphere/issues/5755#issuecomment-632759486
you can config this in you yaml.
```yml
spring.shardingsphere.check.table.metadata.enabled =false
```
if not work,you can update version 4.1.0
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] hermitcai commented on issue #5755: sharding-jdbc slow start
Posted by GitBox <gi...@apache.org>.
hermitcai commented on issue #5755:
URL: https://github.com/apache/shardingsphere/issues/5755#issuecomment-842182059
> this option can avoid sharding tables metadata loading:
>
> > spring.shardingsphere.check.table.metadata.enabled =false
>
> this option can parallelly load metadata for all tables:
>
> > spring.shardingsphere.max.connections.size.per.query =50
thanks, fix my problem.
but my config is:
spring.shardingsphere.props.check.table.metadata.enabled = false
spring.shardingsphere.props.max.connections.size.per.query = 5
version is 4.1.1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] liuqiankun93 commented on issue #5755: sharding-jdbc slow start
Posted by GitBox <gi...@apache.org>.
liuqiankun93 commented on issue #5755:
URL: https://github.com/apache/shardingsphere/issues/5755#issuecomment-632996189
@yu199195
`14:33:42.712 [main] INFO org.apache.shardingsphere.core.log.ConfigurationLogger - Properties:
max.connections.size.per.query: '1'
14:33:42.720 [main] INFO ShardingSphere-metadata - Loading 1 logic tables' meta data.
14:33:43.090 [main] INFO ShardingSphere-metadata - Loading 240 tables' meta data.
14:34:38.056 [main] INFO ShardingSphere-metadata - Meta data load finished, cost 55344 milliseconds.`
`14:36:27.783 [main] INFO org.apache.shardingsphere.core.log.ConfigurationLogger - Properties:
max.connections.size.per.query: '1000'
14:36:27.792 [main] INFO ShardingSphere-metadata - Loading 1 logic tables' meta data.
14:36:28.037 [main] INFO ShardingSphere-metadata - Loading 240 tables' meta data.
14:37:02.237 [main] INFO ShardingSphere-metadata - Meta data load finished, cost 34454 milliseconds.`
when i config the max.connections.size.per.query 1000
it still start slow
so i think it need to not check metadata.But there is no effective when i config the check.table.metadata.enabled
`14:41:08.735 [main] INFO org.apache.shardingsphere.core.log.ConfigurationLogger - Properties:
check.table.metadata.enabled: 'false'
14:41:08.744 [main] INFO ShardingSphere-metadata - Loading 1 logic tables' meta data.
14:41:09.005 [main] INFO ShardingSphere-metadata - Loading 240 tables' meta data.
14:42:05.546 [main] INFO ShardingSphere-metadata - Meta data load finished, cost 56810 milliseconds.`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] liuqiankun93 commented on issue #5755: sharding-jdbc slow start
Posted by GitBox <gi...@apache.org>.
liuqiankun93 commented on issue #5755:
URL: https://github.com/apache/shardingsphere/issues/5755#issuecomment-641775919
@kimmking
my version is 4.1.0 and hand515‘s version is 4.1.1.
check.table.metadata.enabled=false" doesn't work .
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] cws1981 commented on issue #5755: sharding-jdbc slow start
Posted by GitBox <gi...@apache.org>.
cws1981 commented on issue #5755:
URL: https://github.com/apache/shardingsphere/issues/5755#issuecomment-632985823
> @liuqiankun93
> now，we use executerService load table metadata，
> you can config this：
>
> ```
> spring.shardingsphere.max.connections.size.per.query =1 (default 1, changed by you)
> ```
thanks! could I update to 4.1.0 version?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] hand515 commented on issue #5755: sharding-jdbc slow start
Posted by GitBox <gi...@apache.org>.
hand515 commented on issue #5755:
URL: https://github.com/apache/shardingsphere/issues/5755#issuecomment-641183460
Although I updated the version to 4.1.1,it's still slow, and "check.table.metadata.enabled=false" doesn't work .
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] tristaZero commented on issue #5755: sharding-jdbc slow start
Posted by GitBox <gi...@apache.org>.
tristaZero commented on issue #5755:
URL: https://github.com/apache/shardingsphere/issues/5755#issuecomment-698200050
Hi @cws1981 ,
I am sorry to give my comment late. The good news that the performance of starting ShardingProxy has been greatly improved already! Thanks for @xbkaishui, who took much effort in #6212.
Presently, welcome your any test for the `master branch` (Package it local firstly). Your feedback will be the best gift for the community.
Best,
Trista
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] liuqiankun93 commented on issue #5755: sharding-jdbc slow start
Posted by GitBox <gi...@apache.org>.
liuqiankun93 commented on issue #5755:
URL: https://github.com/apache/shardingsphere/issues/5755#issuecomment-632964972
> you can config this in you yaml.
>
> ```yaml
> spring.shardingsphere.check.table.metadata.enabled =false
> ```
>
> if not work,you can update version 4.1.0
i also have this problem.My version is 4.1.0.
`check.table.metadata.enabled (?) | boolean | 是否在启动时检查分表元数据一致性，默认值: false`
The config is default false.But there is no effective.
Then i write this code.it still no effective.
` Properties properties = new Properties();
properties.setProperty("check.table.metadata.enabled","false");`
The load methed cost many time because my database have more than 500 tables.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] cws1981 commented on issue #5755: sharding-jdbc slow start
Posted by GitBox <gi...@apache.org>.
cws1981 commented on issue #5755:
URL: https://github.com/apache/shardingsphere/issues/5755#issuecomment-632755681
> you can not check metadata。
>
>
>
>
?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] hermitcai commented on issue #5755: sharding-jdbc slow start
Posted by GitBox <gi...@apache.org>.
hermitcai commented on issue #5755:
URL: https://github.com/apache/shardingsphere/issues/5755#issuecomment-842182059
> this option can avoid sharding tables metadata loading:
>
> > spring.shardingsphere.check.table.metadata.enabled =false
>
> this option can parallelly load metadata for all tables:
>
> > spring.shardingsphere.max.connections.size.per.query =50
thanks, fix my problem.
but my config is:
spring.shardingsphere.props.check.table.metadata.enabled = false
spring.shardingsphere.props.max.connections.size.per.query = 5
version is 4.1.1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] hand515 edited a comment on issue #5755: sharding-jdbc slow start
Posted by GitBox <gi...@apache.org>.
hand515 edited a comment on issue #5755:
URL: https://github.com/apache/shardingsphere/issues/5755#issuecomment-641183460
Although I updated the version to 4.1.1,it's still slow, and "check.table.metadata.enabled=false" doesn't work .
```
[main] [INFO ConfigurationLogger:104] - Properties:
sql.show: 'true'
check.table.metadata.enabled: 'false'
[main] [INFO ShardingSphere-metadata:131] - Loading 1 logic tables' meta data.
[main] [INFO ShardingSphere-metadata:70] - Loading 348 tables' meta data.
[main] [INFO ShardingSphere-metadata:59] - Meta data load finished, cost 105864 milliseconds.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] yu199195 commented on issue #5755: sharding-jdbc slow start
Posted by GitBox <gi...@apache.org>.
yu199195 commented on issue #5755:
URL: https://github.com/apache/shardingsphere/issues/5755#issuecomment-633368206
@cws1981
You can try this configuration, if not work, You can provide us with your table structure data so that we can optimize it
```yaml
spring.shardingsphere.max.connections.size.per.query =50 (or 100)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] cws1981 edited a comment on issue #5755: sharding-jdbc slow start
Posted by GitBox <gi...@apache.org>.
cws1981 edited a comment on issue #5755:
URL: https://github.com/apache/shardingsphere/issues/5755#issuecomment-633419527
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] cws1981 commented on issue #5755: sharding-jdbc slow start
Posted by GitBox <gi...@apache.org>.
cws1981 commented on issue #5755:
URL: https://github.com/apache/shardingsphere/issues/5755#issuecomment-633419527
> @cws1981
> You can try this configuration, if not work, You can provide us with your table structure data so that we can optimize it
>
> ```yaml
> spring.shardingsphere.max.connections.size.per.query =50 (or 100)
> ```
Thank you for your patience, I will try
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] kimmking commented on issue #5755: sharding-jdbc slow start
Posted by GitBox <gi...@apache.org>.
kimmking commented on issue #5755:
URL: https://github.com/apache/shardingsphere/issues/5755#issuecomment-633637781
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] kimmking closed issue #5755: sharding-jdbc slow start
Posted by GitBox <gi...@apache.org>.
kimmking closed issue #5755:
URL: https://github.com/apache/shardingsphere/issues/5755
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] yu199195 edited a comment on issue #5755: sharding-jdbc slow start
Posted by GitBox <gi...@apache.org>.
yu199195 edited a comment on issue #5755:
URL: https://github.com/apache/shardingsphere/issues/5755#issuecomment-632985170
@liuqiankun93
now，we use executerService load table metadata，
you can config this：
```
spring.shardingsphere.max.connections.size.per.query =1 (default 1, changed by you)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] cws1981 commented on issue #5755: sharding-jdbc slow start
Posted by GitBox <gi...@apache.org>.
cws1981 commented on issue #5755:
URL: https://github.com/apache/shardingsphere/issues/5755#issuecomment-633348111
It is best to have a configuration item that allows me to specify the table for loading metadata
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] kimmking edited a comment on issue #5755: sharding-jdbc slow start
Posted by GitBox <gi...@apache.org>.
kimmking edited a comment on issue #5755:
URL: https://github.com/apache/shardingsphere/issues/5755#issuecomment-633894489
this option can avoid sharding tables metadata loading:
> spring.shardingsphere.check.table.metadata.enabled =false
this option can parallelly load metadata for all tables:
> spring.shardingsphere.max.connections.size.per.query =50
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] cws1981 commented on issue #5755: sharding-jdbc slow start
Posted by GitBox <gi...@apache.org>.
cws1981 commented on issue #5755:
URL: https://github.com/apache/shardingsphere/issues/5755#issuecomment-633353598
A microservice usually has a dozen tables, but the old monolithic system has hundreds of tables.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org