You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2021/01/19 09:31:39 UTC
[GitHub] [shardingsphere] djession opened a new issue #9087: There may be bugs when sharding-proxy integrates with etcd
djession opened a new issue #9087:
URL: https://github.com/apache/shardingsphere/issues/9087
## Bug Report
### Which version of ShardingSphere did you use?
**sharding-proxy alpha 5.0.0**
### Which project did you use? ShardingSphere-JDBC or ShardingSphere-Proxy?
**sharding-proxy alpha 5.0.0**
### Expected behavior
1. initialize the system with the following parameters
```yaml
schemaName: replica_sharding_db
dataSourceCommon:
username: root
password: ding2201
connectionTimeoutMilliseconds: 300000
idleTimeoutMilliseconds: 600000
maxLifetimeMilliseconds: 18000000
maxPoolSize: 50
minPoolSize: 1
maintenanceIntervalMilliseconds: 300000
dataSources:
ds_0:
url: jdbc:mysql://192.168.1.2:3306/demo_ds_0?serverTimezone=UTC&useUnicode=true&characterEncoding=utf-8&useSSL=false&allowPublicKeyRetrieval=true
ds_slave_0:
url: jdbc:mysql://192.168.1.3:3306/demo_ds_0?serverTimezone=UTC&useUnicode=true&characterEncoding=utf-8&useSSL=false&allowPublicKeyRetrieval=true
ds_1:
url: jdbc:mysql://192.168.1.4:3306/demo_ds_1?serverTimezone=UTC&useUnicode=true&characterEncoding=utf-8&useSSL=false&allowPublicKeyRetrieval=true
ds_slave_1:
url: jdbc:mysql://192.168.1.5:3306/demo_ds_1?serverTimezone=UTC&useUnicode=true&characterEncoding=utf-8&useSSL=false&allowPublicKeyRetrieval=true
rules:
- !REPLICA_QUERY
dataSources:
pr_ds_0:
name: pr_ds_0
primaryDataSourceName: ds_0
replicaDataSourceNames:
- ds_slave_0
pr_ds_1:
name: pr_ds_1
primaryDataSourceName: ds_1
replicaDataSourceNames:
- ds_slave_1
- !SHARDING
tables:
t_order:
actualDataNodes: pr_ds_${0..1}.t_order_${0..19}
tableStrategy:
standard:
shardingColumn: order_id
shardingAlgorithmName: t_order_inline
keyGenerateStrategy:
column: order_id
keyGeneratorName: snowflake
t_order_item:
actualDataNodes: pr_ds_${0..1}.t_order_item_${0..19}
tableStrategy:
standard:
shardingColumn: order_id
shardingAlgorithmName: t_order_item_inline
keyGenerateStrategy:
column: order_item_id
keyGeneratorName: snowflake
bindingTables:
- t_order,t_order_item
defaultDatabaseStrategy:
standard:
shardingColumn: user_id
shardingAlgorithmName: database_inline
defaultTableStrategy:
none:
shardingAlgorithms:
database_inline:
type: INLINE
props:
algorithm-expression: pr_ds_${user_id % 2}
allow-range-query-with-inline-sharding: true
t_order_inline:
type: INLINE
props:
algorithm-expression: t_order_${order_id % 19}
allow-range-query-with-inline-sharding: true
t_order_item_inline:
type: INLINE
props:
algorithm-expression: t_order_item_${order_id % 19}
allow-range-query-with-inline-sharding: true
keyGenerators:
snowflake:
type: SNOWFLAKE
props:
worker-id: 123
```
```sql
create table t_order(user_id int not null,order_id bigint not null auto_increment primary key,status varchar(50))engine=innodb default charset=utf8;
```
and then, intput command at the mysql-cli :
`
use replica_sharding_db
show tables
`
all is ok~when i restart the sharding-proxy, at mysql-cli input again:
`
use replica_sharding_db
show tables
`
mysql-cli show the error:
`ERROR 10002 (C1000): 2Unknown exception: [Can not get connection from datasource pr_ds_0.]`
shardingproxy report the error:
`[ERROR] 16:35:01.108 [main] com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Exception during pool initialization.
java.sql.SQLSyntaxErrorException: Unknown database 'demo_ds_0'
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:120)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:836)
at com.mysql.cj.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:456)
at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:246)
at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:198)
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138)
at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:354)
at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:202)
at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:473)
at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:554)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:115)
at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:81)
at org.apache.shardingsphere.proxy.backend.communication.jdbc.datasource.factory.JDBCRawBackendDataSourceFactory.build(JDBCRawBackendDataSourceFactory.java:70)
at org.apache.shardingsphere.proxy.initializer.impl.AbstractBootstrapInitializer.createDataSources(AbstractBootstrapInitializer.java:94)
at org.apache.shardingsphere.proxy.initializer.impl.AbstractBootstrapInitializer.lambda$createDataSourcesMap$0(AbstractBootstrapInitializer.java:88)
at java.util.stream.Collectors.lambda$toMap$58(Collectors.java:1321)
at java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566)
at org.apache.shardingsphere.proxy.initializer.impl.AbstractBootstrapInitializer.createDataSourcesMap(AbstractBootstrapInitializer.java:88)
at org.apache.shardingsphere.proxy.initializer.impl.AbstractBootstrapInitializer.createSchemaContexts(AbstractBootstrapInitializer.java:73)
at org.apache.shardingsphere.proxy.initializer.impl.AbstractBootstrapInitializer.init(AbstractBootstrapInitializer.java:63)
"stdout.log" 65889L, 3179040C`
### Actual behavior
### Reason analyze (If you can)
is there a problem with the format of the data being written to the ETCD that causes the configuration could not repeatable read?
### Steps to reproduce the behavior, such as: SQL to execute, sharding rule configuration, when exception occur etc.
### Example codes for reproduce this issue (such as a github link).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] djession edited a comment on issue #9087: There may be bugs when sharding-proxy integrates with etcd
Posted by GitBox <gi...@apache.org>.
djession edited a comment on issue #9087:
URL: https://github.com/apache/shardingsphere/issues/9087#issuecomment-767389449
@tuohai666 this is my server.yaml
> governance:
> name: governance_ds
> registryCenter:
> type: etcd
> serverLists: http://192.168.183.132:2379
> props:
> retryIntervalMilliseconds: 500
> timeToLiveSeconds: 60
> maxRetries: 3
> operationTimeoutMilliseconds: 500
> overwrite: false
>
> authentication:
> users:
> root:
> password: root
> sharding:
> password: sharding
> authorizedSchemas: sharding_db
>
> props:
> max-connections-size-per-query: 1
> acceptor-size: 16 # The default value is available processors count * 2.
> executor-size: 16 # Infinite by default.
> proxy-frontend-flush-threshold: 128 # The default value is 128.
> proxy-transaction-type: LOCAL
> proxy-opentracing-enabled: false
> proxy-hint-enabled: false
> query-with-cipher-column: true
> sql-show: true
> check-table-metadata-enabled: false
**hi，when i didn't used governance also get the same error**
server.yaml
> authentication:
> users:
> root:
> password: root
> sharding:
> password: sharding
> authorizedSchemas: sharding_db
>
> props:
> max-connections-size-per-query: 1
> acceptor-size: 16 # The default value is available processors count * 2.
> executor-size: 16 # Infinite by default.
> proxy-frontend-flush-threshold: 128 # The default value is 128.
> proxy-transaction-type: LOCAL
> proxy-opentracing-enabled: false
> proxy-hint-enabled: false
> query-with-cipher-column: true
> sql-show: true
> check-table-metadata-enabled: false
> mysql> show tables;
> ERROR 2013 (HY000): Lost connection to MySQL server during query
**this is my test procedure:**
**_1.nitialize the table_**
> mysql> show databases;
> +---------------------+
> | Database |
> +---------------------+
> | replica_sharding_db |
> +---------------------+
> 1 row in set (0.01 sec)
>
> mysql> use replica_sharding_db;
> Database changed
> mysql> create table t_order(user_id int not null,order_id bigint not null auto_increment primary key,status varchar(50))engine=innodb default charset=utf8;
> Query OK, 0 rows affected (0.34 sec)
>
> mysql> show tables;
> +---------------------+
> | Tables_in_demo_ds_0 |
> +---------------------+
> | t_order |
> +---------------------+
> 1 row in set (0.01 sec)
>
> mysql> insert into t_order(user_id,status) values(1,"sdf");
> Query OK, 1 row affected (0.12 sec)
>
> mysql> exit
> Bye
**_2.then reconnect the shardingproxy_**
> mysql -P3307 -h192.168.183.132 -uroot -p
> Enter password:
> Welcome to the MySQL monitor. Commands end with ; or \g.
> Your MySQL connection id is 2
> Server version: 8.0.22-0ubuntu0.20.04.3-ShardingSphere-Proxy 5.0.0-RC1
>
> Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.
>
> Oracle is a registered trademark of Oracle Corporation and/or its
> affiliates. Other names may be trademarks of their respective
> owners.
>
> Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
>
> mysql> show databases;
> +---------------------+
> | Database |
> +---------------------+
> | replica_sharding_db |
> +---------------------+
> 1 row in set (0.00 sec)
>
> mysql> use replica_sharding_db;
> Reading table information for completion of table and column names
> You can turn off this feature to get a quicker startup with -A
>
> Database changed
> mysql> show tables;
> ERROR 2013 (HY000): Lost connection to MySQL server during query
> mysql>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] tuohai666 commented on issue #9087: There may be bugs when sharding-proxy integrates with etcd
Posted by GitBox <gi...@apache.org>.
tuohai666 commented on issue #9087:
URL: https://github.com/apache/shardingsphere/issues/9087#issuecomment-763276437
@djession
1. Please also show server.yaml.
2. Can you try the same procedure without governance and confirm there will be no problem?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] djession closed issue #9087: There may be bugs when sharding-proxy integrates with etcd
Posted by GitBox <gi...@apache.org>.
djession closed issue #9087:
URL: https://github.com/apache/shardingsphere/issues/9087
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] djession commented on issue #9087: There may be bugs when sharding-proxy integrates with etcd
Posted by GitBox <gi...@apache.org>.
djession commented on issue #9087:
URL: https://github.com/apache/shardingsphere/issues/9087#issuecomment-768743789
i've been resloved this issue.
if use mysql-cli connect sharding-proxy must use parameter '-A ',eg:
`mysql -P3307 -h192.168.1.1 -uroot -p -A`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] djession commented on issue #9087: There may be bugs when sharding-proxy integrates with etcd
Posted by GitBox <gi...@apache.org>.
djession commented on issue #9087:
URL: https://github.com/apache/shardingsphere/issues/9087#issuecomment-767389449
> @djession
>
> 1. Please also show server.yaml.
> 2. Can you try the same procedure without governance and confirm there will be no problem?
`yaml
#governance:
# name: governance_ds
# registryCenter:
# type: etcd
# serverLists: http://192.168.183.132:2379
# props:
# retryIntervalMilliseconds: 500
# timeToLiveSeconds: 60
# maxRetries: 3
# operationTimeoutMilliseconds: 500
# overwrite: false
authentication:
users:
root:
password: root
sharding:
password: sharding
authorizedSchemas: sharding_db
props:
max-connections-size-per-query: 1
acceptor-size: 16 # The default value is available processors count * 2.
executor-size: 16 # Infinite by default.
proxy-frontend-flush-threshold: 128 # The default value is 128.
# # LOCAL: Proxy will run with LOCAL transaction.
# # XA: Proxy will run with XA transaction.
# # BASE: Proxy will run with B.A.S.E transaction.
proxy-transaction-type: LOCAL
proxy-opentracing-enabled: false
proxy-hint-enabled: false
query-with-cipher-column: true
sql-show: true
check-table-metadata-enabled: false
`
hi，when i didn't used governance also get the same error
`
mysql> show tables;
ERROR 2013 (HY000): Lost connection to MySQL server during query
`
this is my test procedure:
1.nitialize the table
`
mysql> show databases;
+---------------------+
| Database |
+---------------------+
| replica_sharding_db |
+---------------------+
1 row in set (0.01 sec)
mysql> use replica_sharding_db;
Database changed
mysql> create table t_order(user_id int not null,order_id bigint not null auto_increment primary key,status varchar(50))engine=innodb default charset=utf8;
Query OK, 0 rows affected (0.34 sec)
mysql> show tables;
+---------------------+
| Tables_in_demo_ds_0 |
+---------------------+
| t_order |
+---------------------+
1 row in set (0.01 sec)
mysql> insert into t_order(user_id,status) values(1,"sdf");
Query OK, 1 row affected (0.12 sec)
mysql> exit
Bye
`
2.then reconnect the shardingproxy
`
# mysql -P3307 -h192.168.183.132 -uroot -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 8.0.22-0ubuntu0.20.04.3-ShardingSphere-Proxy 5.0.0-RC1
Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show databases;
+---------------------+
| Database |
+---------------------+
| replica_sharding_db |
+---------------------+
1 row in set (0.00 sec)
mysql> use replica_sharding_db;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> show tables;
ERROR 2013 (HY000): Lost connection to MySQL server during query
mysql>
`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] djession edited a comment on issue #9087: There may be bugs when sharding-proxy integrates with etcd
Posted by GitBox <gi...@apache.org>.
djession edited a comment on issue #9087:
URL: https://github.com/apache/shardingsphere/issues/9087#issuecomment-767389449
this is my server.yaml
> governance:
> name: governance_ds
> registryCenter:
> type: etcd
> serverLists: http://192.168.183.132:2379
> props:
> retryIntervalMilliseconds: 500
> timeToLiveSeconds: 60
> maxRetries: 3
> operationTimeoutMilliseconds: 500
> overwrite: false
>
> authentication:
> users:
> root:
> password: root
> sharding:
> password: sharding
> authorizedSchemas: sharding_db
>
> props:
> max-connections-size-per-query: 1
> acceptor-size: 16 # The default value is available processors count * 2.
> executor-size: 16 # Infinite by default.
> proxy-frontend-flush-threshold: 128 # The default value is 128.
> proxy-transaction-type: LOCAL
> proxy-opentracing-enabled: false
> proxy-hint-enabled: false
> query-with-cipher-column: true
> sql-show: true
> check-table-metadata-enabled: false
**hi，when i didn't used governance also get the same error**
server.yaml
> authentication:
> users:
> root:
> password: root
> sharding:
> password: sharding
> authorizedSchemas: sharding_db
>
> props:
> max-connections-size-per-query: 1
> acceptor-size: 16 # The default value is available processors count * 2.
> executor-size: 16 # Infinite by default.
> proxy-frontend-flush-threshold: 128 # The default value is 128.
> proxy-transaction-type: LOCAL
> proxy-opentracing-enabled: false
> proxy-hint-enabled: false
> query-with-cipher-column: true
> sql-show: true
> check-table-metadata-enabled: false
> mysql> show tables;
> ERROR 2013 (HY000): Lost connection to MySQL server during query
**this is my test procedure:**
**_1.nitialize the table_**
> mysql> show databases;
> +---------------------+
> | Database |
> +---------------------+
> | replica_sharding_db |
> +---------------------+
> 1 row in set (0.01 sec)
>
> mysql> use replica_sharding_db;
> Database changed
> mysql> create table t_order(user_id int not null,order_id bigint not null auto_increment primary key,status varchar(50))engine=innodb default charset=utf8;
> Query OK, 0 rows affected (0.34 sec)
>
> mysql> show tables;
> +---------------------+
> | Tables_in_demo_ds_0 |
> +---------------------+
> | t_order |
> +---------------------+
> 1 row in set (0.01 sec)
>
> mysql> insert into t_order(user_id,status) values(1,"sdf");
> Query OK, 1 row affected (0.12 sec)
>
> mysql> exit
> Bye
**_2.then reconnect the shardingproxy_**
> mysql -P3307 -h192.168.183.132 -uroot -p
> Enter password:
> Welcome to the MySQL monitor. Commands end with ; or \g.
> Your MySQL connection id is 2
> Server version: 8.0.22-0ubuntu0.20.04.3-ShardingSphere-Proxy 5.0.0-RC1
>
> Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.
>
> Oracle is a registered trademark of Oracle Corporation and/or its
> affiliates. Other names may be trademarks of their respective
> owners.
>
> Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
>
> mysql> show databases;
> +---------------------+
> | Database |
> +---------------------+
> | replica_sharding_db |
> +---------------------+
> 1 row in set (0.00 sec)
>
> mysql> use replica_sharding_db;
> Reading table information for completion of table and column names
> You can turn off this feature to get a quicker startup with -A
>
> Database changed
> mysql> show tables;
> ERROR 2013 (HY000): Lost connection to MySQL server during query
> mysql>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org