You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2022/09/21 01:34:12 UTC
[GitHub] [shardingsphere] mxli441 opened a new issue, #21100: shardingsphere-proxy+etcd+opengauss can't recongnize new primary db after switchovering
mxli441 opened a new issue, #21100:
URL: https://github.com/apache/shardingsphere/issues/21100
## Bug Report
**For English only**, other languages will not accept.
Before report a bug, make sure you have:
- Searched open and closed [GitHub issues](https://github.com/apache/shardingsphere/issues).
- Read documentation: [ShardingSphere Doc](https://shardingsphere.apache.org/document/current/en/overview).
Please pay attention on issues you submitted, because we maybe need more details.
If no response anymore and we cannot reproduce it on current information, we will **close it**.
Please answer these questions before submitting your issue. Thanks!
### Which version of ShardingSphere did you use?
shardingsphere-proxy-5.2.0
etcd Version: 3.4.9
opengauss 3.0.0
### Which project did you use? ShardingSphere-JDBC or ShardingSphere-Proxy?
shardingsphere-proxy
### Expected behavior
ShardingSphere-Proxy can recognize new primary db after switchovering
### Actual behavior
ShardingSphere-Proxy can't recognize new primary db after switchovering
### Reason analyze (If you can)
Maybe it is a bug or my configuration is wroing?
### Steps to reproduce the behavior, such as: SQL to execute, sharding rule configuration, when exception occur etc.
### Example codes for reproduce this issue (such as a github link).
=============server.yaml==============
mode:
type: Cluster
repository:
type: Etcd
props:
namespace: /governance_ds
server-lists: http://172.32.153.23x:12379,http://172.32.153.24x:12379,http://172.32.153.23x:12379
retryIntervalMilliseconds: 500
timeToLiveSeconds: 60
maxRetries: 3
- !AUTHORITY
users:
- root@%:root
- sharding@:sharding
provider:
type: ALL_PERMITTED
- !TRANSACTION
defaultType: XA
providerType: Atomikos
- !SQL_PARSER
sqlCommentParseEnabled: true
sqlStatementCache:
initialCapacity: 2000
maximumSize: 65535
parseTreeCache:
initialCapacity: 128
maximumSize: 1024
props:
max-connections-size-per-query: 1
kernel-executor-size: 16 # Infinite by default.
proxy-frontend-flush-threshold: 128 # The default value is 128.
proxy-hint-enabled: false
sql-show: true
===========config-database-discovery.yaml===========
databaseName: readwrite_db
#
dataSources:
ds_0:
url: jdbc:opengauss://172.32.153.24x:31000/d1
username: appuser
password: appuser@1
connectionTimeoutMilliseconds: 3000
idleTimeoutMilliseconds: 60000
maxLifetimeMilliseconds: 1800000
maxPoolSize: 50
minPoolSize: 1
ds_1:
url: jdbc:opengauss://172.32.153.23x:31000/d1
username: appuser
password: appuser@1
connectionTimeoutMilliseconds: 3000
idleTimeoutMilliseconds: 60000
maxLifetimeMilliseconds: 1800000
maxPoolSize: 50
minPoolSize: 1
ds_2:
url: jdbc:opengauss://172.32.153.23X:31000/d1
username: appuser
password: appuser@1
connectionTimeoutMilliseconds: 3000
idleTimeoutMilliseconds: 60000
maxLifetimeMilliseconds: 1800000
maxPoolSize: 50
minPoolSize: 1
rules:
- !DB_DISCOVERY
dataSources:
readwrite_ds:
dataSourceNames:
- ds_0
- ds_1
- ds_2
discoveryHeartbeatName: opengauss-heartbeat
discoveryTypeName: openGauss
discoveryHeartbeats:
opengauss-heartbeat:
props:
keep-alive-cron: '0/5 * * * * ?'
discoveryTypes:
openGauss:
type: openGauss.NORMAL_REPLICATION
props:
group-name: 92504d5b-6dec-11e8-91ea-246e9612aaf1
- !READWRITE_SPLITTING
dataSources:
readwrite_ds:
dynamicStrategy:
autoAwareDataSourceName: readwrite_ds
writeDataSourceQueryEnabled: true
loadBalancerName: random
loadBalancers:
random:
type: ROUND_ROBIN
===============before switchovering ===============
Current cluster topology
+ Cluster: pg_cluster (630762503572893914) --+---------+---------+----+-----------+-----------------+
| Member | Host | Role | State | TL | Lag in MB | Pending restart |
+---------------------+----------------------+---------+---------+----+-----------+-----------------+
| node-172.32.153.23x | 172.32.153.23x:31000 | Replica | running | 1 | 0 | * |
| node-172.32.153.23x | 172.32.153.23x:31000 | Replica | running | 1 | 0 | * |
| node-172.32.153.24x | 172.32.153.24x:31000 | Leader | running | 1 | | * |
+---------------------+----------------------+---------+---------+----+-----------+-----------------+
committed command：etcdctl get --endpoints=127.0.0.1:12379 --prefix /governance_ds/nodes
/governance_ds/nodes/storage_nodes/readwrite_db.readwrite_ds.ds_0
replicationDelayMilliseconds: 0
role: primary
status: enabled
/governance_ds/nodes/storage_nodes/readwrite_db.readwrite_ds.ds_1
replicationDelayMilliseconds: 0
role: member
status: enabled
/governance_ds/nodes/storage_nodes/readwrite_db.readwrite_ds.ds_2
replicationDelayMilliseconds: 0
role: member
status: enabled
inserting can commit
==============After switchovering===============
+ Cluster: pg_cluster (630762503572893914) --+---------+---------+----+-----------+-----------------+
| Member | Host | Role | State | TL | Lag in MB | Pending restart |
+---------------------+----------------------+---------+---------+----+-----------+-----------------+
| node-172.32.153.23x | 172.32.153.23x:31000 | Leader | running | 2 | | * |
| node-172.32.153.23x | 172.32.153.23x:31000 | Replica | running | 2 | 0 | * |
| node-172.32.153.24x | 172.32.153.24x:31000 | Replica | running | 2 | 0 | * |
+---------------------+----------------------+---------+---------+----+-----------+-----------------+
committed command：etcdctl get --endpoints=127.0.0.1:12379 --prefix /governance_ds/nodes 主库无变化
/governance_ds/nodes/storage_nodes/readwrite_db.readwrite_ds.ds_0
replicationDelayMilliseconds: 0
role: primary
status: enabled
/governance_ds/nodes/storage_nodes/readwrite_db.readwrite_ds.ds_1
replicationDelayMilliseconds: 0
role: member
status: enabled
/governance_ds/nodes/storage_nodes/readwrite_db.readwrite_ds.ds_2
replicationDelayMilliseconds: 0
role: member
status: enabled
================Connected to database with port 3307================
gsql -h 172.32.153.23X -p 3307 -U root readwrite_db -r
Password for user root:
gsql ((openGauss 3.0.0 build b592536d) compiled at 2022-04-08 15:30:18 commit 0 last mr )
Non-SSL connection (SSL connection is recommended when requiring high-security)
Type "help" for help.
readwrite_db=> \d
List of relations
Schema | Name | Type | Owner | Storage
--------+------+-------+---------+----------------------------------
public | test | table | appuser | {orientation=row,compression=no}
public | t2 | table | appuser | {orientation=row,compression=no}
public | t1 | table | appuser | {orientation=row,compression=no}
(3 rows)
readwrite_db=> insert into t1 select 1;
ERROR: cannot execute INSERT in a read-only transaction
==I restarted shardingsphere-proxy process then shardingsphere-proxy recognized the new primary db ======
[root@nhgapp05 conf]# ps -ef |grep -i shard
root 55939 1 16 17:31 pts/30 00:00:42 /usr/bin/java -Djava.awt.headless=true -server -Xmx2g -Xms2g -Xmn1g -Xss1m -XX:AutoBoxCacheMax=4096 -XX:+UseNUMA -XX:+DisableExplicitGC -XX:LargePageSizeInBytes=128m -XX:+UseConcMarkSweepGC -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 -Dio.netty.leakDetection.level=DISABLED -classpath /opt/apache-shardingsphere-5.2.0-shardingsphere-proxy-bin/conf:/opt/apache-shardingsphere-5.2.0-shardingsphere-proxy-bin/conf:.:/opt/apache-shardingsphere-5.2.0-shardingsphere-proxy-bin/lib/*:/opt/apache-shardingsphere-5.2.0-shardingsphere-proxy-bin/ext-lib/* org.apache.shardingsphere.proxy.Bootstrap -1 /opt/apache-shardingsphere-5.2.0-shardingsphere-proxy-bin/conf
root 57052 56001 0 17:35 pts/30 00:00:00 grep --color=auto -i shard
[root@nhgapp05 conf]# kill 55939
[root@nhgapp05 conf]# sh ../bin/start.sh
/usr/bin/java
we find java version: java8, full_version=1.8.0_181, full_path=/usr/bin/java
The classpath is /opt/apache-shardingsphere-5.2.0-shardingsphere-proxy-bin/conf:/opt/apache-shardingsphere-5.2.0-shardingsphere-proxy-bin/conf:.:/opt/apache-shardingsphere-5.2.0-shardingsphere-proxy-bin/lib/*:/opt/apache-shardingsphere-5.2.0-shardingsphere-proxy-bin/ext-lib/*
main class org.apache.shardingsphere.proxy.Bootstrap -1 /opt/apache-shardingsphere-5.2.0-shardingsphere-proxy-bin/conf
Starting the ShardingSphere-Proxy ... PID: 57342
Please check the STDOUT file: /opt/apache-shardingsphere-5.2.0-shardingsphere-proxy-bin/logs/stdout.log
===============committed etcdctl command ======================================
etcdctl get --endpoints=127.0.0.1:12379 --prefix /governance_ds/nodes
/governance_ds/nodes/storage_nodes/readwrite_db.readwrite_ds.ds_0
replicationDelayMilliseconds: 0
role: member
status: enabled
/governance_ds/nodes/storage_nodes/readwrite_db.readwrite_ds.ds_1
replicationDelayMilliseconds: 0
role: primary
status: enabled
/governance_ds/nodes/storage_nodes/readwrite_db.readwrite_ds.ds_2
replicationDelayMilliseconds: 0
role: member
status: enabled
============================================================
gsql -h 172.32.153.23x -p 3307 -U root readwrite_db -r
Password for user root:
gsql ((openGauss 3.0.0 build b592536d) compiled at 2022-04-08 15:30:18 commit 0 last mr )
Non-SSL connection (SSL connection is recommended when requiring high-security)
Type "help" for help.
readwrite_db=> insert into t1 select 1;
INSERT 0 1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] zhaojinchao95 commented on issue #21100: shardingsphere-proxy+etcd+opengauss can't recongnize new primary db after switchovering
Posted by GitBox <gi...@apache.org>.
zhaojinchao95 commented on issue #21100:
URL: https://github.com/apache/shardingsphere/issues/21100#issuecomment-1253136703
@mxli441 Hi, thank you for your feedback, the `etcd` does not support the detect of the db, ShardingSphere use `Elastic-job ` to detect db status. So, you can use ZooKeeper and try it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] mxli441 closed issue #21100: shardingsphere-proxy+etcd+opengauss can't recongnize new primary db after switchovering
Posted by GitBox <gi...@apache.org>.
mxli441 closed issue #21100: shardingsphere-proxy+etcd+opengauss can't recongnize new primary db after switchovering
URL: https://github.com/apache/shardingsphere/issues/21100
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] mxli441 commented on issue #21100: shardingsphere-proxy+etcd+opengauss can't recongnize new primary db after switchovering
Posted by GitBox <gi...@apache.org>.
mxli441 commented on issue #21100:
URL: https://github.com/apache/shardingsphere/issues/21100#issuecomment-1253315171
OK,Thank you
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org