You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2020/05/15 08:43:14 UTC

[GitHub] [shardingsphere] menghaoranss opened a new issue #5627: [DISCUSS] ClusterState feature desigin

menghaoranss opened a new issue #5627:
URL: https://github.com/apache/shardingsphere/issues/5627


   Accourding to #5564 
   
   Hi, community
   
   The new module `Control Panel` which plans to consists of `Orchestration`, `Metrics`, `OpenTracing` has been created now. `Control Panel` will provide more powerful features for ShardingSphere in the future. So, we consider designing `ClusterState` module under `Control Panel` to enhance the orchestration feature,  which can monitors the state and heartbeat of Proxy and Datasource in real time.
   
   The following is the design document, any suggestions are welcome. 
   
   ---
   
   #### Current
   
   Currently ShardingSphere only stores and displays Proxy state 
   
   #### Goals
   
   + Add heartbeats between Proxy and corresponding Datasources
   + Real-time monitoring of the connection between Proxy and corresponding Datasources
   + Generate real-time topology of Proxy and Datasources, node online / offline or abnormal can be displayed intuitively
   
   #### Overall Design
   
   ![image-20200515145641071](C:\Users\menghaoran3\AppData\Roaming\Typora\typora-user-images\image-20200515145641071.png)
   
   + Control Panel designs HeartBeat and ClusterState modules, responsible for heartbeat detection and node state processing
   
     + Use scheduled tasks to notify the Proxy of heartbeat detection regularly
     + Configurable whether heartbeat detection is enabled
   
     + Configurable time interval of detection
   
   + Proxy receives heartbeat detection notification from Control Panel and creates heartbeat detection job
   
     + Get a list of Datasources associated with current Proxy
   
     + Get a connection, execute heartbeat detection SQL (configurable)
     + Retry mechanism
       + Configurable whether to retry
       + Configurable maximum number of retries
   
   + Proxy gets Datasource heartbeat detection response
   
   + Proxy notifies Control Panel, update heartbeat to the Registry Center
   
   #### Process
   
   ##### Start Registration
   
   + When the Proxy starts, initialize the Datasource. After the startup is complete, save the Proxy instance and the heartbeat with Datasources  to the Registry Center
   
   ##### Timing Synchronization
   
   + The Control Panel starts scheduled tasks, and regularly notifies all Proxes to perform heartbeat detection
   + Proxy determines whether heartbeat detection is required based on Datasource's last connection timestamp and state
   
   ##### Asynchronous Update
   
   + Once after the Proxy interacts with a Datasource,  update the Datasource last connection timestamp to the Registry Center
   
   #### Storage Structure
   
   `` `
   /
   ├─orchestration-namespace
     ├─orchestration-name 
     │  ├─registry
     │  │  ├─instances
     │  │  │  ├─127.0.0.1@24668@xxx
     │  │  │  │  ├─ds_0
     │  │  │  │  ├─ds_1
     │  │  │  │  ├─ds_2
     │  │  │  │  ├─ds_3
     │  │  │  ├─127.0.0.1@24668@xxx
     │  │  │  │  ├─ds_0
     │  │  │  │  ├─ds_1
     │  │  │  │  ├─ds_2
     │  │  │  │  ├─ds_3
   
   `` `
   
   Datasource node content
   
   `` `
   state = Node state \n
   lastConnect = Timestamp \n
   retryCount = RetryCount
   `` `
   
   + The runtime Datasources are stored under the Proxy instance of the Registry Center
   + When heartbeat response successfully, updates the last connection timestamp and Datasource state to `ONLINE`
   + When heartbeat response failed, updates the Datasource state to `INTERRUPT` and waits for retry
   + New Datasources are added to the Registry Center automatically after detecting heartbeat
   
   #### States
   
   + ONLINE
     + Normal heartbeat response
   + INTERRUPT
     + Heartbeat response failed and does not exceed the maximum number of retries
   + OFFLINE
     + Heartbeat response failed and exceeded the maximum number of retries
   
   #### Configurations
   
   + Heartbeat detection switch
   
     + Enabled by default
   
     + If the switch is closed, keep the current
   
   + Heartbeat detection SQL
   
     + Default `select 1;`
   
   + Heartbeat detection interval
   
     + Default setting `60`s
   
   + Whether to retry
   
     + Enabled by default
   
     + If retry is turned off and the heartbeat response failed, the Datasource state is updated to `OFFLINE`
   
   + Maximum number of retries
   
     + Effective when retry is enabled
     + Default `3`
   
   #### Modules
   
   ```
   /shardingsphere
   ├─control-panel
    |  ├─control-panel-cluster
    |  │  ├─control-panel-cluster-heartbeat
    |  │  ├─control-panle-cluster-state
   ```
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [shardingsphere] menghaoranss commented on issue #5627: [DISCUSS] ClusterState feature design

Posted by GitBox <gi...@apache.org>.
menghaoranss commented on issue #5627:
URL: https://github.com/apache/shardingsphere/issues/5627#issuecomment-630045131


   As mentioned above,there are some `Configuration Items`  need to be configured to decide whether to turn `ClusterState` feature on, whether retry, etc. I recommend we can configure like below in Sharding-Proxy `server.yaml`: 
   ```
   cluster:
      heartBeat:   # heartbeat configuration
         sql: select 1; # heartbeat detection sql
         interval: 60 # heartbeat detection intervalm
         retryEnable: true  # whether to retry on failure
         retryMaximum: 3 # maximum number of retries
   
   proxy.cluster.enable: true  // whether to turn on clusterstate switch
   ```
   
   And now we just need configure `heartBeat`, more configurations about `cluster` should be added under `cluster` in the future.
   
   any suggestions?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [shardingsphere] dongzl commented on issue #5627: [DISCUSS] ClusterState feature design

Posted by GitBox <gi...@apache.org>.
dongzl commented on issue #5627:
URL: https://github.com/apache/shardingsphere/issues/5627#issuecomment-629801052


   Hi @menghaoranss , I understand for the registry center instance, I think use proxy port for the data is more suitable.
   ```
   /
   ├─orchestration-namespace
   ├─orchestration-name
   │ ├─registry
   │ │ ├─instances
   │ │ │ ├─127.0.0.1@3307@xxx
   │ │ │ │ ├─ds_0
   │ │ │ │ ├─ds_1
   │ │ │ │ ├─ds_2
   │ │ │ │ ├─ds_3
   │ │ │ ├─127.0.0.1@3308@xxx
   │ │ │ │ ├─ds_0
   │ │ │ │ ├─ds_1
   │ │ │ │ ├─ds_2
   │ │ │ │ ├─ds_3
   ```
   
   `127.0.0.1@3308@xxx`, IP, Port and other data are more suitable.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [shardingsphere] kimmking commented on issue #5627: [DISCUSS] ClusterState feature design

Posted by GitBox <gi...@apache.org>.
kimmking commented on issue #5627:
URL: https://github.com/apache/shardingsphere/issues/5627#issuecomment-629802033


   I think about @dongzl 's idea, IP+Port is an unique identifier for a proxy, and then process id can be removed.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [shardingsphere] menghaoranss commented on issue #5627: [DISCUSS] ClusterState feature design

Posted by GitBox <gi...@apache.org>.
menghaoranss commented on issue #5627:
URL: https://github.com/apache/shardingsphere/issues/5627#issuecomment-630538034


   ShardingSphere-Proxy support multiple logical data sources, so datasources nodes may be named schema_name.data_source_name, just like: 
   ```
   /
   ├─orchestration-namespace
   ├─orchestration-name
   │ ├─registry
   │ │ ├─instances
   │ │ │ ├─127.0.0.1@3307@xxx
   │ │ │ │ ├─sharding_db.ds_0
   │ │ │ │ ├─sharding_db.ds_1
   │ │ │ │ ├─encrypt_db.ds_2
   │ │ │ │ ├─master_slave_db.ds_3
   │ │ │ ├─127.0.0.1@3308@xxx
   │ │ │ │ ├─sharding_db.ds_0
   │ │ │ │ ├─sharding_db.ds_1
   │ │ │ │ ├─encrypt_db.ds_2
   │ │ │ │ ├─master_slave_db.ds_3
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [shardingsphere] menghaoranss commented on issue #5627: [DISCUSS] ClusterState feature design

Posted by GitBox <gi...@apache.org>.
menghaoranss commented on issue #5627:
URL: https://github.com/apache/shardingsphere/issues/5627#issuecomment-629913635


   @dongzl thanks for your suggestion, i will consider optimizing the structure of registry center instance. 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [shardingsphere] menghaoranss closed issue #5627: [DISCUSS] ClusterState feature design

Posted by GitBox <gi...@apache.org>.
menghaoranss closed issue #5627:
URL: https://github.com/apache/shardingsphere/issues/5627


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [shardingsphere] menghaoranss commented on issue #5627: [DISCUSS] ClusterState feature design

Posted by GitBox <gi...@apache.org>.
menghaoranss commented on issue #5627:
URL: https://github.com/apache/shardingsphere/issues/5627#issuecomment-631338056


   Because the zookeeper EPHEMERAL node does not support child nodes,so,we will persist Datasource state to `instance` node:
   ```
   instanceState: DISABLE
   datasources:
       sharding_db.ds_1:
           state: Node state
           lastConnect: Timestamp
           retryCount: RetryCount
       sharding_db.ds_2:
           state: Node state
           lastConnect: Timestamp
           retryCount: RetryCount	
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org