You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2020/12/03 10:17:01 UTC
[GitHub] [shardingsphere] iamif3000 opened a new issue #8494: [bug] no response package during performance test
iamif3000 opened a new issue #8494:
URL: https://github.com/apache/shardingsphere/issues/8494
### Which version of ShardingSphere did you use?
tag v4.1.1
### Which project did you use? ShardingSphere-JDBC or ShardingSphere-Proxy?
sharding-proxy
### Expected behavior
The performance test could run successfully.
### Actual behavior
The test client just hang(waiting for package from sharding-proxy).
### Reason analyze (If you can)
I think the mysql protocol implementation or the network layer of sharding-proxy has a bug or bugs.
### Steps to reproduce the behavior, such as: SQL to execute, sharding rule configuration, when exception occur etc.
```
my test env:
3 physic-machine (40core128G)
1 for mariadb 10.0.2
1 for sharding-proxy
1 for jmeter
the table definition:
CREATE TABLE `jmeter_test` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`field01` varchar(100) NOT NULL DEFAULT '',
`field02` varchar(100) NOT NULL DEFAULT '',
`field03` varchar(100) NOT NULL DEFAULT '',
`field04` varchar(100) NOT NULL DEFAULT '',
`field05` varchar(100) NOT NULL DEFAULT '',
`field06` varchar(100) NOT NULL DEFAULT '',
`field07` varchar(100) NOT NULL DEFAULT '',
`field08` varchar(100) NOT NULL DEFAULT '',
`field09` varchar(100) NOT NULL DEFAULT '',
`field10` varchar(100) NOT NULL DEFAULT '',
`field11` varchar(100) NOT NULL DEFAULT '',
`field12` varchar(100) NOT NULL DEFAULT '',
`field13` varchar(100) NOT NULL DEFAULT '',
`field14` varchar(100) NOT NULL DEFAULT '',
`field15` varchar(100) NOT NULL DEFAULT '',
`field16` varchar(100) NOT NULL DEFAULT '',
`field17` varchar(100) NOT NULL DEFAULT '',
`field18` varchar(100) NOT NULL DEFAULT '',
`field19` varchar(100) NOT NULL DEFAULT '',
`field20` varchar(100) NOT NULL DEFAULT '',
`field21` varchar(100) NOT NULL DEFAULT '',
`field22` varchar(100) NOT NULL DEFAULT '',
`field23` varchar(100) NOT NULL DEFAULT '',
`field24` varchar(100) NOT NULL DEFAULT '',
`field25` varchar(100) NOT NULL DEFAULT '',
`create_by` bigint(20) NOT NULL DEFAULT 0,
`c_t` bigint(20) NOT NULL DEFAULT 0,
`update_by` bigint(20) NOT NULL DEFAULT 0,
`u_t` bigint(20) NOT NULL DEFAULT 0,
PRIMARY KEY (`id`),
KEY `idx_u_t` (`u_t`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
the insert sql:
insert into jmeter_test values (null, now(), now(), now(), now(), now(), now(), now(), now(), now(), now(), now(), now(), now(), now(), now(), now(), now(), now(), now(), now(), now(), now(), now(), now(), now(), unix_timestamp(), unix_timestamp(), unix_timestamp(), unix_timestamp());
the query sql:
select id,field01,field02,field03,field04,field05,field06,field07,field08,field09,field10,field11,field12,field13,field14,field15,field16,field17,field18,field19,field20,field21,field22,field23,field24,field25,create_by,c_t,update_by,u_t from jmeter_test order by id desc limit 100;
the configuration of sharding-proxy:
server.yaml --- start
authentication:
users:
root:
password: root
sharding:
password: sharding
authorizedSchemas: master_slave_db
props:
max.connections.size.per.query: 1
acceptor.size: 160
executor.size: 160
proxy.frontend.flush.threshold: 128
proxy.transaction.type: LOCAL
proxy.opentracing.enabled: false
proxy.hint.enabled: false
query.with.cipher.column: true
# 测读写分离的时候设置为true，便于查看sql路由
sql.show: false
allow.range.query.with.inline.sharding: false
server.yaml --- end
config-master_slave.yaml --- start
schemaName: master_slave_db
dataSources:
master_ds:
url: jdbc:mysql://10.12.37.64:4455/proxy?serverTimezone=UTC&useSSL=false
username: proxy
password: p123456
connectionTimeoutMilliseconds: 30000
idleTimeoutMilliseconds: 60000
maxLifetimeMilliseconds: 1800000
maxPoolSize: 1000
minPoolSize: 1000
masterSlaveRule:
name: ms_ds
masterDataSourceName: master_ds
slaveDataSourceNames:
- master_ds
config-master_slave.yaml --- end
You may use jmeter for the test. For me I found jmeter's own performance is not good enough. I wrote two simple test programs.
```
[select.txt](https://github.com/apache/shardingsphere/files/5635303/select.txt)
[insert.txt](https://github.com/apache/shardingsphere/files/5635305/insert.txt)
```
To compile and run the two programs, you have to download boost(don't have to compile, mysql connector just uses the header files) and mysql connector for c++(1.1.9 works).
Compile and update shell env
g++ select.cpp -oselect -O2 -I./mysql/include -I../boost -lpthread -lmysqlcppconn -L./mysql/lib
g++ insert.cpp -o insert -O2 -I./mysql/include -I../boost -lpthread -lmysqlcppconn -L./mysql/lib
export LD_LIBRARY_PATH=/home/xxxx/mysql/lib:$LD_LIBRARY_PATH
When running "select", you'll see the test program juse hang. And this picture explains what happend.
```
```
The test program is waiting for proxy's response.
```
### Example codes for reproduce this issue (such as a github link).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] tristaZero commented on issue #8494: [bug] no response package during performance test
Posted by GitBox <gi...@apache.org>.
tristaZero commented on issue #8494:
URL: https://github.com/apache/shardingsphere/issues/8494#issuecomment-738598406
Hi @iamif3000 ,
Thanks for your performance test.
1. Could you give [Performance Test](https://github.com/apache/shardingsphere/issues/6536) a look? This issue showed you the relevant tests we did. I guess that will give you some hints.
2. Could you test the latest version 5.0.0-alpha? We fixed a few bugs and provided other great features in this release, worthy of your consideration of online usage.
3. Does any of the queries cause this issue showed in the image?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] iamif3000 commented on issue #8494: [bug] no response package during performance test
Posted by GitBox <gi...@apache.org>.
iamif3000 commented on issue #8494:
URL: https://github.com/apache/shardingsphere/issues/8494#issuecomment-739460875
@tristaZero
I made a test on 5.0.0-alpha. The test can be done. This bug is not reproducible on the latest alpha version.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] tristaZero commented on issue #8494: [bug] no response package during performance test
Posted by GitBox <gi...@apache.org>.
tristaZero commented on issue #8494:
URL: https://github.com/apache/shardingsphere/issues/8494#issuecomment-739721763
@iamif3000 Hence, do you think it is time to close this issue? Thx.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] tristaZero closed issue #8494: [bug] no response package during performance test
Posted by GitBox <gi...@apache.org>.
tristaZero closed issue #8494:
URL: https://github.com/apache/shardingsphere/issues/8494
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] iamif3000 commented on issue #8494: [bug] no response package during performance test
Posted by GitBox <gi...@apache.org>.
iamif3000 commented on issue #8494:
URL: https://github.com/apache/shardingsphere/issues/8494#issuecomment-739897738
@tristaZero
Yes. Thanks for your fast reply.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] iamif3000 commented on issue #8494: [bug] no response package during performance test
Posted by GitBox <gi...@apache.org>.
iamif3000 commented on issue #8494:
URL: https://github.com/apache/shardingsphere/issues/8494#issuecomment-738603110
@tristaZero
I'll make a test on 5.0.0-alpha.
And about the third question, the query is simply ""select id,field01,field02,field03,field04,field05,field06,field07,field08,field09,field10,field11,field12,field13,field14,field15,field16,field17,field18,field19,field20,field21,field22,field23,field24,field25,create_by,c_t,update_by,u_t from jmeter_test order by id desc limit 100;"
You could read the first part of this post and see how I made the test.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org