You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shenyu.apache.org by de...@apache.org on 2021/09/21 09:59:38 UTC
[incubator-shenyu-website] branch main updated: zookeeper data sync
source code analysis (#285)
This is an automated email from the ASF dual-hosted git repository.
dengliming pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-shenyu-website.git
The following commit(s) were added to refs/heads/main by this push:
new c5b7b1e zookeeper data sync source code analysis (#285)
c5b7b1e is described below
commit c5b7b1ea77d6bcd5d66f19f094435917513e455b
Author: midnight2104 <33...@qq.com>
AuthorDate: Tue Sep 21 17:59:35 2021 +0800
zookeeper data sync source code analysis (#285)
* zookeeper data sync source code analysis
* fix md lint
---
blog/SourceCode-Analysis-WebSocket-Data-Sync.md | 7 +-
blog/SourceCode-Analysis-ZooKeeper-Data-Sync.md | 824 +++++++++++++++++++++
.../SourceCode-Analysis-WebSocket-Data-Sync.md | 2 +-
.../SourceCode-Analysis-ZooKeeper-Data-Sync.md | 818 ++++++++++++++++++++
.../data-changed-listener.png | Bin 0 -> 16959 bytes
.../handler-selector.png | Bin 0 -> 123655 bytes
.../update-selector-en.png | Bin 0 -> 77176 bytes
.../update-selector-zh.png | Bin 0 -> 85385 bytes
.../zk-sync-sequence-admin-en.png | Bin 0 -> 58003 bytes
.../zk-sync-sequence-admin-zh.png | Bin 0 -> 62818 bytes
.../zk-sync-sequence-gateway-en.png | Bin 0 -> 47685 bytes
.../zk-sync-sequence-gateway-zh.png | Bin 0 -> 49165 bytes
.../zookeeper-node.png | Bin 0 -> 65862 bytes
13 files changed, 1646 insertions(+), 5 deletions(-)
diff --git a/blog/SourceCode-Analysis-WebSocket-Data-Sync.md b/blog/SourceCode-Analysis-WebSocket-Data-Sync.md
index 4660e02..eb6adfd 100644
--- a/blog/SourceCode-Analysis-WebSocket-Data-Sync.md
+++ b/blog/SourceCode-Analysis-WebSocket-Data-Sync.md
@@ -1,7 +1,7 @@
---
slug: code-analysis-websocket-data-sync
title: WebSocket Data Synchronization Source Code Analysis
-author: Liu Liang
+author: midnight2104
author_title: Apache ShenYu Committer
author_url: https://github.com/midnight2104
tags: [websocket,data sync,Apache ShenYu]
@@ -649,8 +649,7 @@ It has no additional logic and calls the `subscribeDataHandler()` method directl
```java
/**
- * The universal plug-in data subscriber, responsible for handling all plug-in, selector, and rule information
- * The type Common plugin data subscriber.
+ * The common plugin data subscriber, responsible for handling all plug-in, selector, and rule information
*/
public class CommonPluginDataSubscriber implements PluginDataSubscriber {
//......
@@ -774,7 +773,7 @@ Second, if each plugin has its own processing logic, then do it. Through the `ID

-After the above source tracing, and through a practical case, in the 'admin' side to add a selector data, will `websocket` data synchronization process analysis cleared.
+After the above source tracing, and through a practical case, in the `admin` side to add a selector data, will `websocket` data synchronization process analysis cleared.
Let's use the following figure to concatenate the data synchronization process on the gateway side:
diff --git a/blog/SourceCode-Analysis-ZooKeeper-Data-Sync.md b/blog/SourceCode-Analysis-ZooKeeper-Data-Sync.md
new file mode 100644
index 0000000..3fd9ef1
--- /dev/null
+++ b/blog/SourceCode-Analysis-ZooKeeper-Data-Sync.md
@@ -0,0 +1,824 @@
+---
+slug: code-analysis-zookeeper-data-sync
+title: ZooKeeper Data Synchronization Source Code Analysis
+author: midnight2104
+author_title: Apache ShenYu Committer
+author_url: https://github.com/midnight2104
+tags: [zookeeper,data sync,Apache ShenYu]
+---
+
+> [Apache ShenYu](https://shenyu.apache.org/zh/docs/index) is an asynchronous, high-performance, cross-language, responsive API gateway.
+
+In `ShenYu` gateway, data synchronization refers to how to synchronize the updated data to the gateway after the data is sent in the background management system. The Apache ShenYu gateway currently supports data synchronization for `ZooKeeper`, `WebSocket`, `http long poll`, `Nacos`, `etcd` and `Consul`. The main content of this article is based on `WebSocket` data synchronization source code analysis.
+
+> This paper based on `shenyu-2.4.0` version of the source code analysis, the official website of the introduction of please refer to the [Data Synchronization Design](https://shenyu.apache.org/docs/design/data-sync/) .
+
+### 1. About ZooKeeper
+
+Apache ZooKeeper is a software project of the Apache Software Foundation that provides open source distributed configuration services, synchronization services, and naming registries for large-scale distributed computing. ZooKeeper nodes store their data in a hierarchical namespace, much like a file system or a prefix tree structure. Clients can read and write on nodes and thus have a shared configuration service in this way.
+
+### 2. Admin Data Sync
+
+We traced the source code from a real case, such as updating a selector data in the `Divide` plugin to a weight of 90 in a background administration system:
+
+
+
+#### 2.1 Accept Data
+
+- SelectorController.createSelector()
+
+Enter the createSelector() method of the `SelectorController` class, which validates data, adds or updates data, and returns results.
+
+```java
+@Validated
+@RequiredArgsConstructor
+@RestController
+@RequestMapping("/selector")
+public class SelectorController {
+
+ @PutMapping("/{id}")
+ public ShenyuAdminResult updateSelector(@PathVariable("id") final String id, @Valid @RequestBody final SelectorDTO selectorDTO) {
+ // set the current selector data ID
+ selectorDTO.setId(id);
+ // create or update operation
+ Integer updateCount = selectorService.createOrUpdate(selectorDTO);
+ // return result
+ return ShenyuAdminResult.success(ShenyuResultMessage.UPDATE_SUCCESS, updateCount);
+ }
+
+ // ......
+}
+```
+
+#### 2.2 Handle Data
+
+- SelectorServiceImpl.createOrUpdate()
+
+Convert data in the `SelectorServiceImpl` class using the `createOrUpdate()` method, save it to the database, publish the event, update `upstream`.
+
+```java
+@RequiredArgsConstructor
+@Service
+public class SelectorServiceImpl implements SelectorService {
+ // eventPublisher
+ private final ApplicationEventPublisher eventPublisher;
+
+ @Override
+ @Transactional(rollbackFor = Exception.class)
+ public int createOrUpdate(final SelectorDTO selectorDTO) {
+ int selectorCount;
+ // build data DTO --> DO
+ SelectorDO selectorDO = SelectorDO.buildSelectorDO(selectorDTO);
+ List<SelectorConditionDTO> selectorConditionDTOs = selectorDTO.getSelectorConditions();
+ // insert or update ?
+ if (StringUtils.isEmpty(selectorDTO.getId())) {
+ // insert into data
+ selectorCount = selectorMapper.insertSelective(selectorDO);
+ // insert into condition data
+ selectorConditionDTOs.forEach(selectorConditionDTO -> {
+ selectorConditionDTO.setSelectorId(selectorDO.getId());
+ selectorConditionMapper.insertSelective(SelectorConditionDO.buildSelectorConditionDO(selectorConditionDTO));
+ });
+ // check selector add
+ if (dataPermissionMapper.listByUserId(JwtUtils.getUserInfo().getUserId()).size() > 0) {
+ DataPermissionDTO dataPermissionDTO = new DataPermissionDTO();
+ dataPermissionDTO.setUserId(JwtUtils.getUserInfo().getUserId());
+ dataPermissionDTO.setDataId(selectorDO.getId());
+ dataPermissionDTO.setDataType(AdminConstants.SELECTOR_DATA_TYPE);
+ dataPermissionMapper.insertSelective(DataPermissionDO.buildPermissionDO(dataPermissionDTO));
+ }
+
+ } else {
+ // update data, delete and then insert
+ selectorCount = selectorMapper.updateSelective(selectorDO);
+ //delete rule condition then add
+ selectorConditionMapper.deleteByQuery(new SelectorConditionQuery(selectorDO.getId()));
+ selectorConditionDTOs.forEach(selectorConditionDTO -> {
+ selectorConditionDTO.setSelectorId(selectorDO.getId());
+ SelectorConditionDO selectorConditionDO = SelectorConditionDO.buildSelectorConditionDO(selectorConditionDTO);
+ selectorConditionMapper.insertSelective(selectorConditionDO);
+ });
+ }
+ // publish event
+ publishEvent(selectorDO, selectorConditionDTOs);
+
+ // update upstream
+ updateDivideUpstream(selectorDO);
+ return selectorCount;
+ }
+
+ // ......
+
+}
+```
+
+In the `Service` class to persist data, i.e. to the database, this should be familiar, not expand. The update upstream operation is analyzed in the corresponding section below, focusing on the publish event operation, which performs data synchronization.
+
+The logic of the `publishEvent()` method is to find the plugin corresponding to the selector, build the conditional data, and publish the change data.
+
+```java
+ private void publishEvent(final SelectorDO selectorDO, final List<SelectorConditionDTO> selectorConditionDTOs) {
+ // find plugin of selector
+ PluginDO pluginDO = pluginMapper.selectById(selectorDO.getPluginId());
+ // build condition data
+ List<ConditionData> conditionDataList = selectorConditionDTOs.stream().map(ConditionTransfer.INSTANCE::mapToSelectorDTO).collect(Collectors.toList());
+ // publish event
+ eventPublisher.publishEvent(new DataChangedEvent(ConfigGroupEnum.SELECTOR, DataEventTypeEnum.UPDATE,
+ Collections.singletonList(SelectorDO.transFrom(selectorDO, pluginDO.getName(), conditionDataList))));
+ }
+```
+
+Change data released by `eventPublisher.PublishEvent()` is complete, the `eventPublisher` object is a `ApplicationEventPublisher` class, The fully qualified class name is `org.springframework.context.ApplicationEventPublisher`. Here we see that publishing data is done through `Spring` related functionality.
+
+> `ApplicationEventPublisher`:
+>
+> When a state change, the publisher calls `ApplicationEventPublisher` of `publishEvent` method to release an event, `Spring` container broadcast event for all observers, The observer's `onApplicationEvent` method is called to pass the event object to the observer. There are two ways to call `publishEvent` method, one is to implement the interface by the container injection `ApplicationEventPublisher` object and then call the method, the other is a direct call container, the method of tw [...]
+>
+> - `ApplicationEventPublisher`: publish event;
+> - `ApplicationEvent`: `Spring` event, record the event source, time, and data;
+> - `ApplicationListener`: event listener, observer.
+
+In Spring event publishing mechanism, there are three objects,
+
+An object is a publish event `ApplicationEventPublisher`, in `ShenYu` through the constructor in the injected a `eventPublisher`.
+
+The other object is `ApplicationEvent` , inherited from `ShenYu` through `DataChangedEvent`, representing the event object.
+
+```java
+public class DataChangedEvent extends ApplicationEvent {
+//......
+}
+```
+
+The last object is `ApplicationListener` in `ShenYu` in through `DataChangedEventDispatcher` class implements this interface, as the event listener, responsible for handling the event object.
+
+```java
+@Component
+public class DataChangedEventDispatcher implements ApplicationListener<DataChangedEvent>, InitializingBean {
+
+ //......
+
+}
+```
+
+#### 2.3 Dispatch Data
+
+- DataChangedEventDispatcher.onApplicationEvent()
+
+Released when the event is completed, will automatically enter the `DataChangedEventDispatcher` class `onApplicationEvent()` method of handling events.
+
+```java
+@Component
+public class DataChangedEventDispatcher implements ApplicationListener<DataChangedEvent>, InitializingBean {
+
+ /**
+ * This method is called when there are data changes
+ * @param event
+ */
+ @Override
+ @SuppressWarnings("unchecked")
+ public void onApplicationEvent(final DataChangedEvent event) {
+ // Iterate through the data change listener (usually using a data synchronization approach is fine)
+ for (DataChangedListener listener : listeners) {
+ // What kind of data has changed
+ switch (event.getGroupKey()) {
+ case APP_AUTH: // app auth data
+ listener.onAppAuthChanged((List<AppAuthData>) event.getSource(), event.getEventType());
+ break;
+ case PLUGIN: // plugin data
+ listener.onPluginChanged((List<PluginData>) event.getSource(), event.getEventType());
+ break;
+ case RULE: // rule data
+ listener.onRuleChanged((List<RuleData>) event.getSource(), event.getEventType());
+ break;
+ case SELECTOR: // selector data
+ listener.onSelectorChanged((List<SelectorData>) event.getSource(), event.getEventType());
+ break;
+ case META_DATA: // metadata
+ listener.onMetaDataChanged((List<MetaData>) event.getSource(), event.getEventType());
+ break;
+ default: // other types throw exception
+ throw new IllegalStateException("Unexpected value: " + event.getGroupKey());
+ }
+ }
+ }
+
+}
+```
+
+When there is a data change, the `onApplicationEvent` method is called and all the data change listeners are iterated to determine the data type and handed over to the appropriate data listener for processing.
+
+ShenYu groups all the data into five categories: `APP_AUTH`, `PLUGIN`, `RULE`, `SELECTOR` and `META_DATA`.
+
+Here the data change listener (`DataChangedListener`) is an abstraction of the data synchronization policy. Its concrete implementation is:
+
+
+
+These implementation classes are the synchronization strategies currently supported by ShenYu:
+
+- `WebsocketDataChangedListener`: data synchronization based on Websocket;
+- `ZookeeperDataChangedListener`:data synchronization based on Zookeeper;
+- `ConsulDataChangedListener`: data synchronization based on Consul;
+- `EtcdDataDataChangedListener`:data synchronization based on etcd;
+- `HttpLongPollingDataChangedListener`:data synchronization based on http long polling;
+- `NacosDataChangedListener`:data synchronization based on nacos;
+
+Given that there are so many implementation strategies, how do you decide which to use?
+
+Because this paper is based on `zookeeper` data synchronization source code analysis, so here to `ZookeeperDataChangedListener` as an example, the analysis of how it is loaded and implemented.
+
+A global search in the source code project shows that its implementation is done in the `DataSyncConfiguration` class.
+
+```java
+/**
+ * Data Sync Configuration
+ * By springboot conditional assembly
+ * The type Data sync configuration.
+ */
+@Configuration
+public class DataSyncConfiguration {
+
+
+ /**
+ * zookeeper data sunc
+ * The type Zookeeper listener.
+ */
+ @Configuration
+ @ConditionalOnProperty(prefix = "shenyu.sync.zookeeper", name = "url") // The condition property is loaded only when it is met
+ @Import(ZookeeperConfiguration.class)
+ static class ZookeeperListener {
+
+ /**
+ * Config event listener data changed listener.
+ * @param zkClient the zk client
+ * @return the data changed listener
+ */
+ @Bean
+ @ConditionalOnMissingBean(ZookeeperDataChangedListener.class)
+ public DataChangedListener zookeeperDataChangedListener(final ZkClient zkClient) {
+ return new ZookeeperDataChangedListener(zkClient);
+ }
+
+ /**
+ * Zookeeper data init zookeeper data init.
+ * @param zkClient the zk client
+ * @param syncDataService the sync data service
+ * @return the zookeeper data init
+ */
+ @Bean
+ @ConditionalOnMissingBean(ZookeeperDataInit.class)
+ public ZookeeperDataInit zookeeperDataInit(final ZkClient zkClient, final SyncDataService syncDataService) {
+ return new ZookeeperDataInit(zkClient, syncDataService);
+ }
+ }
+
+ // other code is omitted......
+}
+
+```
+
+This configuration class is implemented through the SpringBoot conditional assembly class. The ZookeeperListener class has several annotations:
+
+- `@Configuration`: Configuration file, application context;
+
+- `@ConditionalOnProperty(prefix = "shenyu.sync.zookeeper", name = "url")`: attribute condition. The configuration class takes effect only when the condition is met. That is, when we have the following configuration, `ZooKeeper` is used for data synchronization.
+
+ ```properties
+ shenyu:
+ sync:
+ zookeeper:
+ url: localhost:2181
+ sessionTimeout: 5000
+ connectionTimeout: 2000
+ ```
+
+- `@Import(ZookeeperConfiguration.class)`:import `ZookeeperConfiguration`;
+
+```java
+ @EnableConfigurationProperties(ZookeeperProperties.class) // enable zookeeper properties
+ public class ZookeeperConfiguration {
+
+ /**
+ * register zkClient in spring ioc.
+ * @param zookeeperProp the zookeeper configuration
+ * @return ZkClient {@linkplain ZkClient}
+ */
+ @Bean
+ @ConditionalOnMissingBean(ZkClient.class)
+ public ZkClient zkClient(final ZookeeperProperties zookeeperProp) {
+ return new ZkClient(zookeeperProp.getUrl(), zookeeperProp.getSessionTimeout(), zookeeperProp.getConnectionTimeout()); // 读取zk配置信息,并创建zkClient
+ }
+ }
+```
+
+```java
+@Data
+@ConfigurationProperties(prefix = "shenyu.sync.zookeeper") // zookeeper properties
+public class ZookeeperProperties {
+
+ private String url;
+
+ private Integer sessionTimeout;
+
+ private Integer connectionTimeout;
+
+ private String serializer;
+}
+```
+
+When we take the initiative to configuration, use the `zookeeper` data synchronization, `zookeeperDataChangedListener` is generated. So in the event handler `onApplicationEvent()`, it goes to the corresponding `listener`. In our case, it is a selector data update, data synchronization is `zookeeper`, so, the code will enter the `ZookeeperDataChangedListener` selector data change process.
+
+```java
+ @Override
+ @SuppressWarnings("unchecked")
+ public void onApplicationEvent(final DataChangedEvent event) {
+ // Iterate through the data change listener (usually using a data synchronization approach is fine)
+ for (DataChangedListener listener : listeners) {
+ // what kind of data has changed
+ switch (event.getGroupKey()) {
+
+ // other code logic is omitted
+
+ case SELECTOR: // selector data
+ listener.onSelectorChanged((List<SelectorData>) event.getSource(), event.getEventType()); // In our case, will enter the ZookeeperDataChangedListener selector data change process
+ break;
+ }
+ }
+```
+
+#### 2.4 Zookeeper Data Changed Listener
+
+- ZookeeperDataChangedListener.onSelectorChanged()
+
+In the `onSelectorChanged()` method, determine the type of action, whether to refresh synchronization or update or create synchronization. Determine whether the node is in `zk` based on the current selector data.
+
+```java
+
+/**
+ * use ZooKeeper to publish change data
+ */
+public class ZookeeperDataChangedListener implements DataChangedListener {
+
+ // The selector information changed
+ @Override
+ public void onSelectorChanged(final List<SelectorData> changed, final DataEventTypeEnum eventType) {
+ // refresh
+ if (eventType == DataEventTypeEnum.REFRESH && !changed.isEmpty()) {
+ String selectorParentPath = DefaultPathConstants.buildSelectorParentPath(changed.get(0).getPluginName());
+ deleteZkPathRecursive(selectorParentPath);
+ }
+ // changed data
+ for (SelectorData data : changed) {
+ // build selector real path
+ String selectorRealPath = DefaultPathConstants.buildSelectorRealPath(data.getPluginName(), data.getId());
+ // delete
+ if (eventType == DataEventTypeEnum.DELETE) {
+ deleteZkPath(selectorRealPath);
+ continue;
+ }
+ // selector parent path
+ String selectorParentPath = DefaultPathConstants.buildSelectorParentPath(data.getPluginName());
+ // create parent node
+ createZkNode(selectorParentPath);
+ // insert or update data
+ insertZkNode(selectorRealPath, data);
+ }
+ }
+
+ // create zk node
+ private void createZkNode(final String path) {
+ // create only if it does not exist
+ if (!zkClient.exists(path)) {
+ zkClient.createPersistent(path, true);
+ }
+ }
+
+ // insert zk node
+ private void insertZkNode(final String path, final Object data) {
+ // create zk node
+ createZkNode(path);
+ // write data by zkClient
+ zkClient.writeData(path, null == data ? "" : GsonUtils.getInstance().toJson(data));
+ }
+
+}
+```
+
+As long as the changed data is correctly written to the `zk` node, the `admin` side of the operation is complete. `ShenYu` uses `zk` for data synchronization, `zk` nodes are carefully designed.
+
+In our current case, updating one of the selector data in the `Divide` plugin with a weight of 90 updates specific nodes in the graph.
+
+
+
+
+We series the above update flow with a sequence diagram.
+
+
+
+
+
+### 3. Gateway Data Sync
+
+Assume that the ShenYu gateway is already running properly, and the data synchronization mode is also `Zookeeper`. How does the gateway receive and process the selector data after updating it on the admin side and sending the changed data to ZK? Let's continue our source code analysis to find out.
+
+#### 3.1 ZkClient Accept Data
+
+- ZkClient.subscribeDataChanges()
+
+There is a `ZookeeperSyncDataService` class on the gateway, which subscribing to the data node through `ZkClient` and can sense when the data changes.
+
+
+```java
+/**
+ * ZookeeperSyncDataService
+ */
+public class ZookeeperSyncDataService implements SyncDataService, AutoCloseable {
+
+private void subscribeSelectorDataChanges(final String path) {
+ // zkClient subscribe data
+ zkClient.subscribeDataChanges(path, new IZkDataListener() {
+ @Override
+ public void handleDataChange(final String dataPath, final Object data) {
+ cacheSelectorData(GsonUtils.getInstance().fromJson(data.toString(), SelectorData.class)); // zk node data changed
+ }
+
+ @Override
+ public void handleDataDeleted(final String dataPath) {
+ unCacheSelectorData(dataPath); // zk node data deleted
+ }
+ });
+ }
+
+ // ...
+}
+```
+
+ZooKeeper's `Watch` mechanism notifies subscribing clients of node changes. In our case, updating the selector information goes to the `handleDataChange()` method. `cacheSelectorData()` is used to process data.
+
+
+#### 3.2 Handle Data
+
+- ZookeeperSyncDataService.cacheSelectorData()
+
+The data is not null, and caching the selector data is again handled by `PluginDataSubscriber`.
+
+```java
+ private void cacheSelectorData(final SelectorData selectorData) {
+ Optional.ofNullable(selectorData)
+ .ifPresent(data -> Optional.ofNullable(pluginDataSubscriber).ifPresent(e -> e.onSelectorSubscribe(data)));
+ }
+```
+
+`PluginDataSubscriber` is an interface, it is only a `CommonPluginDataSubscriber` implementation class, responsible for data processing plugin, selector and rules.
+
+
+
+#### 3.3 Common Plugin Data Subscriber
+
+- PluginDataSubscriber.onSelectorSubscribe()
+
+It has no additional logic and calls the `subscribeDataHandler()` method directly. Within methods, there are data types (plugins, selectors, or rules) and action types (update or delete) to perform different logic.
+
+
+```java
+/**
+ * The common plugin data subscriber, responsible for handling all plug-in, selector, and rule information
+ */
+public class CommonPluginDataSubscriber implements PluginDataSubscriber {
+ //......
+ // handle selector data
+ @Override
+ public void onSelectorSubscribe(final SelectoData selectorData) {
+ subscribeDataHandler(selectorData, DataEventTypeEnum.UPDATE);
+ }
+
+ // A subscription data handler that handles updates or deletions of data
+ private <T> void subscribeDataHandler(final T classData, final DataEventTypeEnum dataType) {
+ Optional.ofNullable(classData).ifPresent(data -> {
+ // plugin data
+ if (data instanceof PluginData) {
+ PluginData pluginData = (PluginData) data;
+ if (dataType == DataEventTypeEnum.UPDATE) { // update
+ // save the data to gateway memory
+ BaseDataCache.getInstance().cachePluginData(pluginData);
+ // If each plugin has its own processing logic, then do it
+ Optional.ofNullable(handlerMap.get(pluginData.getName())).ifPresent(handler -> handler.handlerPlugin(pluginData));
+ } else if (dataType == DataEventTypeEnum.DELETE) { // delete
+ // delete the data from gateway memory
+ BaseDataCache.getInstance().removePluginData(pluginData);
+ // If each plugin has its own processing logic, then do it
+ Optional.ofNullable(handlerMap.get(pluginData.getName())).ifPresent(handler -> handler.removePlugin(pluginData));
+ }
+ } else if (data instanceof SelectorData) { // selector data
+ SelectorData selectorData = (SelectorData) data;
+ if (dataType == DataEventTypeEnum.UPDATE) { // update
+ // save the data to gateway memory
+ BaseDataCache.getInstance().cacheSelectData(selectorData);
+ // If each plugin has its own processing logic, then do it
+ Optional.ofNullable(handlerMap.get(selectorData.getPluginName())).ifPresent(handler -> handler.handlerSelector(selectorData));
+ } else if (dataType == DataEventTypeEnum.DELETE) { // delete
+ // delete the data from gateway memory
+ BaseDataCache.getInstance().removeSelectData(selectorData);
+ // If each plugin has its own processing logic, then do it
+ Optional.ofNullable(handlerMap.get(selectorData.getPluginName())).ifPresent(handler -> handler.removeSelector(selectorData));
+ }
+ } else if (data instanceof RuleData) { // rule data
+ RuleData ruleData = (RuleData) data;
+ if (dataType == DataEventTypeEnum.UPDATE) { // update
+ // save the data to gateway memory
+ BaseDataCache.getInstance().cacheRuleData(ruleData);
+ // If each plugin has its own processing logic, then do it
+ Optional.ofNullable(handlerMap.get(ruleData.getPluginName())).ifPresent(handler -> handler.handlerRule(ruleData));
+ } else if (dataType == DataEventTypeEnum.DELETE) { // delete
+ // delete the data from gateway memory
+ BaseDataCache.getInstance().removeRuleData(ruleData);
+ // If each plugin has its own processing logic, then do it
+ Optional.ofNullable(handlerMap.get(ruleData.getPluginName())).ifPresent(handler -> handler.removeRule(ruleData));
+ }
+ }
+ });
+ }
+
+}
+```
+
+#### 3.4 Data cached to Memory
+
+
+Adding a selector will enter the following logic:
+
+```java
+// save the data to gateway memory
+BaseDataCache.getInstance().cacheSelectData(selectorData);
+// If each plugin has its own processing logic, then do it
+Optional.ofNullable(handlerMap.get(selectorData.getPluginName())).ifPresent(handler -> handler.handlerSelector(selectorData));
+```
+
+One is to save the data to the gateway's memory. BaseDataCache is the class that ultimately caches data, implemented in a singleton pattern. The selector data is stored in the `SELECTOR_MAP` Map. In the subsequent use, also from this data.
+
+
+```java
+public final class BaseDataCache {
+ // private instance
+ private static final BaseDataCache INSTANCE = new BaseDataCache();
+ // private constructor
+ private BaseDataCache() {
+ }
+
+ /**
+ * Gets instance.
+ * public method
+ * @return the instance
+ */
+ public static BaseDataCache getInstance() {
+ return INSTANCE;
+ }
+
+ /**
+ * A Map of the cache selector data
+ * pluginName -> SelectorData.
+ */
+ private static final ConcurrentMap<String, List<SelectorData>> SELECTOR_MAP = Maps.newConcurrentMap();
+
+ public void cacheSelectData(final SelectorData selectorData) {
+ Optional.ofNullable(selectorData).ifPresent(this::selectorAccept);
+ }
+
+ /**
+ * cache selector data.
+ * @param data the selector data
+ */
+ private void selectorAccept(final SelectorData data) {
+ String key = data.getPluginName();
+ if (SELECTOR_MAP.containsKey(key)) { // Update operation, delete before insert
+ List<SelectorData> existList = SELECTOR_MAP.get(key);
+ final List<SelectorData> resultList = existList.stream().filter(r -> !r.getId().equals(data.getId())).collect(Collectors.toList());
+ resultList.add(data);
+ final List<SelectorData> collect = resultList.stream().sorted(Comparator.comparing(SelectorData::getSort)).collect(Collectors.toList());
+ SELECTOR_MAP.put(key, collect);
+ } else { // Add new operations directly to Map
+ SELECTOR_MAP.put(key, Lists.newArrayList(data));
+ }
+ }
+
+}
+```
+
+Second, if each plugin has its own processing logic, then do it. Through the `IDEA` editor, you can see that after adding a selector, there are the following plugins and processing. We're not going to expand it here.
+
+
+
+After the above source tracking, and through a practical case, in the `admin` end to update a selector data, the `ZooKeeper` data synchronization process analysis is clear.
+
+
+Let's series the data synchronization process on the gateway side through the sequence diagram:
+
+
+
+
+The data synchronization process has been analyzed. In order to prevent the synchronization process from being interrupted, other logic is ignored during the analysis. We also need to analyze the process of Admin synchronization data initialization and gateway synchronization operation initialization.
+
+
+### 4. Admin Data Sync initialization
+
+When `admin` starts, the current data will be fully synchronized to `zk`, the implementation logic is as follows:
+
+
+```java
+
+/**
+ * Zookeeper data init
+ */
+public class ZookeeperDataInit implements CommandLineRunner {
+
+ private final ZkClient zkClient;
+
+ private final SyncDataService syncDataService;
+
+ /**
+ * Instantiates a new Zookeeper data init.
+ *
+ * @param zkClient the zk client
+ * @param syncDataService the sync data service
+ */
+ public ZookeeperDataInit(final ZkClient zkClient, final SyncDataService syncDataService) {
+ this.zkClient = zkClient;
+ this.syncDataService = syncDataService;
+ }
+
+ @Override
+ public void run(final String... args) {
+ String pluginPath = DefaultPathConstants.PLUGIN_PARENT;
+ String authPath = DefaultPathConstants.APP_AUTH_PARENT;
+ String metaDataPath = DefaultPathConstants.META_DATA;
+ // Determine whether data exists in zk
+ if (!zkClient.exists(pluginPath) && !zkClient.exists(authPath) && !zkClient.exists(metaDataPath)) {
+ syncDataService.syncAll(DataEventTypeEnum.REFRESH);
+ }
+ }
+}
+
+```
+
+Check whether there is data in `zk`, if not, then synchronize.
+
+
+`ZookeeperDataInit` implements the `CommandLineRunner` interface. It is an interface provided by `SpringBoot` that executes the `run()` method after all `Spring Beans` initializations and is often used for initialization operations in a project.
+
+
+- SyncDataService.syncAll()
+
+Query data from the database, and then perform full data synchronization, all authentication information, plugin information, selector information, rule information, and metadata information. Synchronous events are published primarily through `eventPublisher`. After publishing the event via `publishEvent()`, the `ApplicationListener` performs the event change operation. In `ShenYu` is mentioned in `DataChangedEventDispatcher`.
+
+```java
+@Service
+public class SyncDataServiceImpl implements SyncDataService {
+ // eventPublisher
+ private final ApplicationEventPublisher eventPublisher;
+
+ /***
+ * sync all data
+ * @param type the type
+ * @return
+ */
+ @Override
+ public boolean syncAll(final DataEventTypeEnum type) {
+ // app auth data
+ appAuthService.syncData();
+ // plugin data
+ List<PluginData> pluginDataList = pluginService.listAll();
+ eventPublisher.publishEvent(new DataChangedEvent(ConfigGroupEnum.PLUGIN, type, pluginDataList));
+ // selector data
+ List<SelectorData> selectorDataList = selectorService.listAll();
+ eventPublisher.publishEvent(new DataChangedEvent(ConfigGroupEnum.SELECTOR, type, selectorDataList));
+ // rule data
+ List<RuleData> ruleDataList = ruleService.listAll();
+ eventPublisher.publishEvent(new DataChangedEvent(ConfigGroupEnum.RULE, type, ruleDataList));
+ // metadata
+ metaDataService.syncData();
+ return true;
+ }
+
+}
+```
+
+### 5. Gateway Data Sync Init
+
+The initial operation of data synchronization on the gateway side is mainly the node in the subscription `zk`. When there is a data change, the changed data will be received. This relies on the `Watch` mechanism of `ZooKeeper`. In `ShenYu`, the one responsible for `zk` data synchronization is `ZookeeperSyncDataService`, also mentioned earlier.
+
+The function logic of `ZookeeperSyncDataService` is completed in the process of instantiation: the subscription to `Shenyu` data synchronization node in `zk` is completed. Subscription here is divided into two kinds, one kind is existing node data updated above, through this `zkClient.subscribeDataChanges()` method; Another kind is under the current node, add or delete nodes change namely child nodes, it through `zkClient.subscribeChildChanges()` method.
+
+`ZookeeperSyncDataService` code is a bit too much, here we use plugin data read and subscribe to track, other types of data operation principle is the same.
+
+
+```java
+
+/**
+ * zookeeper sync data service
+ */
+public class ZookeeperSyncDataService implements SyncDataService, AutoCloseable {
+ // At instantiation time, the data is read from the ZK and the node is subscribed
+ public ZookeeperSyncDataService(/* omit the construction argument */ ) {
+ this.zkClient = zkClient;
+ this.pluginDataSubscriber = pluginDataSubscriber;
+ this.metaDataSubscribers = metaDataSubscribers;
+ this.authDataSubscribers = authDataSubscribers;
+ // watch plugin, selector and rule data
+ watcherData();
+ // watch app auth data
+ watchAppAuth();
+ // watch metadata
+ watchMetaData();
+ }
+
+ private void watcherData() {
+ // plugin node path
+ final String pluginParent = DefaultPathConstants.PLUGIN_PARENT;
+ // all plugin nodes
+ List<String> pluginZKs = zkClientGetChildren(pluginParent);
+ for (String pluginName : pluginZKs) {
+ // watch plugin, selector, rule data node
+ watcherAll(pluginName);
+ }
+ //subscribing to child nodes (adding or removing a plugin)
+ zkClient.subscribeChildChanges(pluginParent, (parentPath, currentChildren) -> {
+ if (CollectionUtils.isNotEmpty(currentChildren)) {
+ for (String pluginName : currentChildren) {
+ // you need to subscribe to all plugin, selector, and rule data for the child node
+ watcherAll(pluginName);
+ }
+ }
+ });
+ }
+
+ private void watcherAll(final String pluginName) {
+ // watch plugin
+ watcherPlugin(pluginName);
+ // watch selector
+ watcherSelector(pluginName);
+ // watch rule
+ watcherRule(pluginName);
+ }
+
+ private void watcherPlugin(final String pluginName) {
+ // plugin path
+ String pluginPath = DefaultPathConstants.buildPluginPath(pluginName);
+ // create if not exist
+ if (!zkClient.exists(pluginPath)) {
+ zkClient.createPersistent(pluginPath, true);
+ }
+ // read the current node data on zk and deserialize it
+ PluginData pluginData = null == zkClient.readData(pluginPath) ? null
+ : GsonUtils.getInstance().fromJson((String) zkClient.readData(pluginPath), PluginData.class);
+ // cached into gateway memory
+ cachePluginData(pluginData);
+ // subscribe plugin data
+ subscribePluginDataChanges(pluginPath, pluginName);
+ }
+
+ private void cachePluginData(final PluginData pluginData) {
+ //omit implementation logic, is actually the CommonPluginDataSubscriber operation, can connect with the front
+ }
+
+ private void subscribePluginDataChanges(final String pluginPath, final String pluginName) {
+ // subscribe data changes
+ zkClient.subscribeDataChanges(pluginPath, new IZkDataListener() {
+
+ @Override
+ public void handleDataChange(final String dataPath, final Object data) { // update
+ //omit implementation logic, is actually the CommonPluginDataSubscriber operation, can connect with the front
+ }
+
+ @Override
+ public void handleDataDeleted(final String dataPath) { // delete
+ // Omit implementation logic, is actually the CommonPluginDataSubscriber operation, can connect with the front
+
+ }
+ });
+ }
+
+}
+```
+
+The above source code is given comments, I believe you can understand. The main logic for subscribing to plug-in data is as follows:
+
+> 1. Create the current plugin path
+> 2. Create a path if it does not exist
+> 3. Read the current node data on zK and deserialize it
+> 4. The plugin data is cached in the gateway memory
+> 5. Subscribe to the plug-in node
+
+
+### 6. Summary
+
+This paper through a practical case, `Zookeeper` data synchronization principle source code analysis. The main knowledge points involved are as follows:
+
+- Data synchronization based on `ZooKeeper` is mainly implemented through `watch` mechanism;
+
+- Complete event publishing and listening via `Spring`;
+
+- Support multiple synchronization strategies through abstract `DataChangedListener` interface, interface oriented programming;
+
+- Use singleton design pattern to cache data class `BaseDataCache`;
+
+- Loading of configuration classes via conditional assembly of `SpringBoot` and `starter` loading mechanism.
diff --git a/i18n/zh/docusaurus-plugin-content-blog/SourceCode-Analysis-WebSocket-Data-Sync.md b/i18n/zh/docusaurus-plugin-content-blog/SourceCode-Analysis-WebSocket-Data-Sync.md
index 845d7a7..9e0ccd6 100644
--- a/i18n/zh/docusaurus-plugin-content-blog/SourceCode-Analysis-WebSocket-Data-Sync.md
+++ b/i18n/zh/docusaurus-plugin-content-blog/SourceCode-Analysis-WebSocket-Data-Sync.md
@@ -1,7 +1,7 @@
---
slug: code-analysis-websocket-data-sync
title: WebSocket数据同步源码分析
-author: Liu Liang
+author: midnight2104
author_title: Apache ShenYu Committer
author_url: https://github.com/midnight2104
tags: [websocket,data sync,Apache ShenYu]
diff --git a/i18n/zh/docusaurus-plugin-content-blog/SourceCode-Analysis-ZooKeeper-Data-Sync.md b/i18n/zh/docusaurus-plugin-content-blog/SourceCode-Analysis-ZooKeeper-Data-Sync.md
new file mode 100644
index 0000000..b4ab77a
--- /dev/null
+++ b/i18n/zh/docusaurus-plugin-content-blog/SourceCode-Analysis-ZooKeeper-Data-Sync.md
@@ -0,0 +1,818 @@
+---
+slug: code-analysis-zookeeper-data-sync
+title: ZooKeeper数据同步源码分析
+author: midnight2104
+author_title: Apache ShenYu Committer
+author_url: https://github.com/midnight2104
+tags: [zookeeper,data sync,Apache ShenYu]
+---
+
+> [Apache ShenYu](https://shenyu.apache.org/zh/docs/index) 是一个异步的,高性能的,跨语言的,响应式的 `API` 网关。
+
+在`ShenYu`网关中,数据同步是指,当在后台管理系统中,数据发送了更新后,如何将更新的数据同步到网关中。`Apache ShenYu` 网关当前支持`ZooKeeper`、`WebSocket`、`Http长轮询`、`Nacos` 、`Etcd` 和 `Consul` 进行数据同步。本文的主要内容是基于`ZooKeeper`的数据同步源码分析。
+
+> 本文基于`shenyu-2.4.0`版本进行源码分析,官网的介绍请参考 [数据同步原理](https://shenyu.apache.org/zh/docs/design/data-sync) 。
+
+### 1. 关于ZooKeeper
+
+[`Apache ZooKeeper`](https://zh.wikipedia.org/wiki/Apache_ZooKeeper)是`Apache`软件基金会的一个软件项目,它为大型分布式计算提供开源的分布式配置服务、同步服务和命名注册。`ZooKeeper`节点将它们的数据存储于一个分层的名字空间,非常类似于一个文件系统或一个前缀树结构。客户端可以在节点读写,从而以这种方式拥有一个共享的配置服务。
+
+### 2. Admin数据同步
+
+我们从一个实际案例进行源码追踪,比如在后台管理系统中,对`Divide`插件中的一条选择器数据进行更新,将权重更新为90:
+
+
+
+#### 2.1 接收数据
+
+- SelectorController.createSelector()
+
+进入`SelectorController`类中的`updateSelector()`方法,它负责数据的校验,添加或更新数据,返回结果信息。
+
+```java
+@Validated
+@RequiredArgsConstructor
+@RestController
+@RequestMapping("/selector")
+public class SelectorController {
+
+ @PutMapping("/{id}")
+ public ShenyuAdminResult updateSelector(@PathVariable("id") final String id, @Valid @RequestBody final SelectorDTO selectorDTO) {
+ // 设置当前选择器数据id
+ selectorDTO.setId(id);
+ // 创建或更新操作
+ Integer updateCount = selectorService.createOrUpdate(selectorDTO);
+ // 返回结果信息
+ return ShenyuAdminResult.success(ShenyuResultMessage.UPDATE_SUCCESS, updateCount);
+ }
+
+ // ......
+}
+```
+
+#### 2.2 处理数据
+
+- SelectorServiceImpl.createOrUpdate()
+
+在`SelectorServiceImpl`类中通过`createOrUpdate()`方法完成数据的转换,保存到数据库,发布事件,更新`upstream`。
+
+```java
+@RequiredArgsConstructor
+@Service
+public class SelectorServiceImpl implements SelectorService {
+ // 负责事件发布的eventPublisher
+ private final ApplicationEventPublisher eventPublisher;
+
+ @Override
+ @Transactional(rollbackFor = Exception.class)
+ public int createOrUpdate(final SelectorDTO selectorDTO) {
+ int selectorCount;
+ // 构建数据 DTO --> DO
+ SelectorDO selectorDO = SelectorDO.buildSelectorDO(selectorDTO);
+ List<SelectorConditionDTO> selectorConditionDTOs = selectorDTO.getSelectorConditions();
+ // 判断是添加还是更新
+ if (StringUtils.isEmpty(selectorDTO.getId())) {
+ // 插入选择器数据
+ selectorCount = selectorMapper.insertSelective(selectorDO);
+ // 插入选择器中的条件数据
+ selectorConditionDTOs.forEach(selectorConditionDTO -> {
+ selectorConditionDTO.setSelectorId(selectorDO.getId());
+ selectorConditionMapper.insertSelective(SelectorConditionDO.buildSelectorConditionDO(selectorConditionDTO));
+ });
+ // check selector add
+ // 权限检查
+ if (dataPermissionMapper.listByUserId(JwtUtils.getUserInfo().getUserId()).size() > 0) {
+ DataPermissionDTO dataPermissionDTO = new DataPermissionDTO();
+ dataPermissionDTO.setUserId(JwtUtils.getUserInfo().getUserId());
+ dataPermissionDTO.setDataId(selectorDO.getId());
+ dataPermissionDTO.setDataType(AdminConstants.SELECTOR_DATA_TYPE);
+ dataPermissionMapper.insertSelective(DataPermissionDO.buildPermissionDO(dataPermissionDTO));
+ }
+
+ } else {
+ // 更新数据,先删除再新增
+ selectorCount = selectorMapper.updateSelective(selectorDO);
+ //delete rule condition then add
+ selectorConditionMapper.deleteByQuery(new SelectorConditionQuery(selectorDO.getId()));
+ selectorConditionDTOs.forEach(selectorConditionDTO -> {
+ selectorConditionDTO.setSelectorId(selectorDO.getId());
+ SelectorConditionDO selectorConditionDO = SelectorConditionDO.buildSelectorConditionDO(selectorConditionDTO);
+ selectorConditionMapper.insertSelective(selectorConditionDO);
+ });
+ }
+ // 发布事件
+ publishEvent(selectorDO, selectorConditionDTOs);
+
+ // 更新upstream
+ updateDivideUpstream(selectorDO);
+ return selectorCount;
+ }
+
+
+ // ......
+
+}
+
+```
+
+在`Serrvice`类完成数据的持久化操作,即保存数据到数据库,这个比较简单,就不深入追踪了。关于更新`upstream`操作,放到后面对应的章节中进行分析,重点关注发布事件的操作,它会执行数据同步。
+
+`publishEvent()`方法的逻辑是:找到选择器对应的插件,构建条件数据,发布变更数据。
+
+```java
+ private void publishEvent(final SelectorDO selectorDO, final List<SelectorConditionDTO> selectorConditionDTOs) {
+ // 找到选择器对应的插件
+ PluginDO pluginDO = pluginMapper.selectById(selectorDO.getPluginId());
+ // 构建条件数据
+ List<ConditionData> conditionDataList = selectorConditionDTOs.stream().map(ConditionTransfer.INSTANCE::mapToSelectorDTO).collect(Collectors.toList());
+ // 发布变更数据
+ eventPublisher.publishEvent(new DataChangedEvent(ConfigGroupEnum.SELECTOR, DataEventTypeEnum.UPDATE,
+ Collections.singletonList(SelectorDO.transFrom(selectorDO, pluginDO.getName(), conditionDataList))));
+ }
+```
+
+发布变更数据通过`eventPublisher.publishEvent()`完成,这个`eventPublisher`对象是一个`ApplicationEventPublisher`类,这个类的全限定名是`org.springframework.context.ApplicationEventPublisher`。看到这儿,我们知道了发布数据是通过`Spring`相关的功能来完成的。
+
+> 关于`ApplicationEventPublisher`:
+>
+> 当有状态发生变化时,发布者调用 `ApplicationEventPublisher` 的 `publishEvent` 方法发布一个事件,`Spring`容器广播事件给所有观察者,调用观察者的 `onApplicationEvent` 方法把事件对象传递给观察者。调用 `publishEvent`方法有两种途径,一种是实现接口由容器注入 `ApplicationEventPublisher` 对象然后调用其方法,另一种是直接调用容器的方法,两种方法发布事件没有太大区别。
+>
+> - `ApplicationEventPublisher`:发布事件;
+> - `ApplicationEvent`:`Spring` 事件,记录事件源、时间和数据;
+> - `ApplicationListener`:事件监听者,观察者;
+
+
+在`Spring`的事件发布机制中,有三个对象,
+
+一个是发布事件的`ApplicationEventPublisher`,在`ShenYu`中通过构造器注入了一个`eventPublisher`。
+
+另一个对象是`ApplicationEvent`,在`ShenYu`中通过`DataChangedEvent`继承了它,表示事件对象。
+
+```java
+public class DataChangedEvent extends ApplicationEvent {
+//......
+}
+```
+
+最后一个是 `ApplicationListener`,在`ShenYu`中通过`DataChangedEventDispatcher`类实现了该接口,作为事件的监听者,负责处理事件对象。
+
+```java
+@Component
+public class DataChangedEventDispatcher implements ApplicationListener<DataChangedEvent>, InitializingBean {
+
+ //......
+
+}
+```
+
+#### 2.3 分发数据
+
+- DataChangedEventDispatcher.onApplicationEvent()
+
+当事件发布完成后,会自动进入到`DataChangedEventDispatcher`类中的`onApplicationEvent()`方法,进行事件处理。
+
+```java
+@Component
+public class DataChangedEventDispatcher implements ApplicationListener<DataChangedEvent>, InitializingBean {
+
+ /**
+ * 有数据变更时,调用此方法
+ * @param event
+ */
+ @Override
+ @SuppressWarnings("unchecked")
+ public void onApplicationEvent(final DataChangedEvent event) {
+ // 遍历数据变更监听器(一般使用一种数据同步的方式就好了)
+ for (DataChangedListener listener : listeners) {
+ // 哪种数据发生变更
+ switch (event.getGroupKey()) {
+ case APP_AUTH: // 认证信息
+ listener.onAppAuthChanged((List<AppAuthData>) event.getSource(), event.getEventType());
+ break;
+ case PLUGIN: // 插件信息
+ listener.onPluginChanged((List<PluginData>) event.getSource(), event.getEventType());
+ break;
+ case RULE: // 规则信息
+ listener.onRuleChanged((List<RuleData>) event.getSource(), event.getEventType());
+ break;
+ case SELECTOR: // 选择器信息
+ listener.onSelectorChanged((List<SelectorData>) event.getSource(), event.getEventType());
+ break;
+ case META_DATA: // 元数据
+ listener.onMetaDataChanged((List<MetaData>) event.getSource(), event.getEventType());
+ break;
+ default: // 其他类型,抛出异常
+ throw new IllegalStateException("Unexpected value: " + event.getGroupKey());
+ }
+ }
+ }
+
+}
+```
+
+当有数据变更时,调用`onApplicationEvent`方法,然后遍历所有数据变更监听器,判断是哪种数据类型,交给相应的数据监听器进行处理。
+
+`ShenYu`将所有数据进行了分组,一共是五种:认证信息、插件信息、规则信息、选择器信息和元数据。
+
+这里的数据变更监听器(`DataChangedListener`),就是数据同步策略的抽象,它的具体实现有:
+
+
+
+这几个实现类就是当前`ShenYu`支持的同步策略:
+
+- `WebsocketDataChangedListener`:基于`websocket`的数据同步;
+- `ZookeeperDataChangedListener`:基于`zookeeper`的数据同步;
+- `ConsulDataChangedListener`:基于`consul`的数据同步;
+- `EtcdDataDataChangedListener`:基于`etcd`的数据同步;
+- `HttpLongPollingDataChangedListener`:基于`http长轮询`的数据同步;
+- `NacosDataChangedListener`:基于`nacos`的数据同步;
+
+既然有这么多种实现策略,那么如何确定使用哪一种呢?
+
+因为本文是基于`Zookeeper`的数据同步源码分析,所以这里以`ZookeeperDataChangedListener`为例,分析它是如何被加载并实现的。
+
+通过在源码工程中进行全局搜索,可以看到,它的实现是在`DataSyncConfiguration`类完成的。
+
+```java
+/**
+ * 数据同步配置类
+ * 通过springboot条件装配实现
+ * The type Data sync configuration.
+ */
+@Configuration
+public class DataSyncConfiguration {
+
+
+ /**
+ * zookeeper数据同步
+ * The type Zookeeper listener.
+ */
+ @Configuration
+ @ConditionalOnProperty(prefix = "shenyu.sync.zookeeper", name = "url") // 条件属性,满足才会被加载
+ @Import(ZookeeperConfiguration.class)
+ static class ZookeeperListener {
+
+ /**
+ * Config event listener data changed listener.
+ * 创建Zookeeper数据变更监听器
+ * @param zkClient the zk client
+ * @return the data changed listener
+ */
+ @Bean
+ @ConditionalOnMissingBean(ZookeeperDataChangedListener.class)
+ public DataChangedListener zookeeperDataChangedListener(final ZkClient zkClient) {
+ return new ZookeeperDataChangedListener(zkClient);
+ }
+
+ /**
+ * Zookeeper data init zookeeper data init.
+ * 创建 Zookeeper 数据初始化类
+ * @param zkClient the zk client
+ * @param syncDataService the sync data service
+ * @return the zookeeper data init
+ */
+ @Bean
+ @ConditionalOnMissingBean(ZookeeperDataInit.class)
+ public ZookeeperDataInit zookeeperDataInit(final ZkClient zkClient, final SyncDataService syncDataService) {
+ return new ZookeeperDataInit(zkClient, syncDataService);
+ }
+ }
+
+ //省略了其他代码......
+}
+
+```
+
+这个配置类是通过`SpringBoot`条件装配类实现的。在`ZookeeperListener`类上面有几个注解:
+
+- `@Configuration`:配置文件,应用上下文;
+
+- `@ConditionalOnProperty(prefix = "shenyu.sync.zookeeper", name = "url")`:属性条件判断,满足条件,该配置类才会生效。也就是说,当我们有如下配置时,就会采用`zookeeper`进行数据同步。
+
+ ```properties
+ shenyu:
+ sync:
+ zookeeper:
+ url: localhost:2181
+ sessionTimeout: 5000
+ connectionTimeout: 2000
+ ```
+
+- `@Import(ZookeeperConfiguration.class)`:导入另一个类`ZookeeperConfiguration`;
+
+```java
+ @EnableConfigurationProperties(ZookeeperProperties.class) // 启用zk属性配置类
+ public class ZookeeperConfiguration {
+
+ /**
+ * register zkClient in spring ioc.
+ * 向 Spring IOC 容器注册 zkClient
+ * @param zookeeperProp the zookeeper configuration
+ * @return ZkClient {@linkplain ZkClient}
+ */
+ @Bean
+ @ConditionalOnMissingBean(ZkClient.class)
+ public ZkClient zkClient(final ZookeeperProperties zookeeperProp) {
+ return new ZkClient(zookeeperProp.getUrl(), zookeeperProp.getSessionTimeout(), zookeeperProp.getConnectionTimeout()); // 读取zk配置信息,并创建zkClient
+ }
+ }
+```
+
+```java
+@Data
+@ConfigurationProperties(prefix = "shenyu.sync.zookeeper") // zk属性配置
+public class ZookeeperProperties {
+
+ private String url;
+
+ private Integer sessionTimeout;
+
+ private Integer connectionTimeout;
+
+ private String serializer;
+}
+```
+
+当我们主动配置,采用`zookeeper`进行数据同步时,`zookeeperDataChangedListener`就会生成。所以在事件处理方法`onApplicationEvent()`中,就会到相应的`listener`中。在我们的案例中,是对一条选择器数据进行更新,数据同步采用的是`zookeeper`,所以,代码会进入到`ZookeeperDataChangedListener`进行选择器数据变更处理。
+
+```java
+ @Override
+ @SuppressWarnings("unchecked")
+ public void onApplicationEvent(final DataChangedEvent event) {
+ // 遍历数据变更监听器(一般使用一种数据同步的方式就好了)
+ for (DataChangedListener listener : listeners) {
+ // 哪种数据发生变更
+ switch (event.getGroupKey()) {
+
+ // 省略了其他逻辑
+
+ case SELECTOR: // 选择器信息
+ listener.onSelectorChanged((List<SelectorData>) event.getSource(), event.getEventType()); // 在我们的案例中,会进入到ZookeeperDataChangedListener进行选择器数据变更处理
+ break;
+ }
+ }
+```
+
+#### 2.4 Zookeeper数据变更监听器
+
+- ZookeeperDataChangedListener.onSelectorChanged()
+
+ 在`onSelectorChanged()`方法中,判断操作类型,是刷新同步还是更新或创建同步。根据当前选择器数据信息判断节点是否在`zk`中。
+
+```java
+
+/**
+ * 使用 zookeeper 发布变更数据
+ */
+public class ZookeeperDataChangedListener implements DataChangedListener {
+
+ // 选择器信息发生改变
+ @Override
+ public void onSelectorChanged(final List<SelectorData> changed, final DataEventTypeEnum eventType) {
+ // 刷新操作
+ if (eventType == DataEventTypeEnum.REFRESH && !changed.isEmpty()) {
+ String selectorParentPath = DefaultPathConstants.buildSelectorParentPath(changed.get(0).getPluginName());
+ deleteZkPathRecursive(selectorParentPath);
+ }
+ // 发生变更的数据
+ for (SelectorData data : changed) {
+ // 构建选择器数据的真实路径
+ String selectorRealPath = DefaultPathConstants.buildSelectorRealPath(data.getPluginName(), data.getId());
+ // 如果是删除操作
+ if (eventType == DataEventTypeEnum.DELETE) {
+ // 删除当前数据
+ deleteZkPath(selectorRealPath);
+ continue;
+ }
+ // 父节点路径
+ String selectorParentPath = DefaultPathConstants.buildSelectorParentPath(data.getPluginName());
+ // 创建父节点
+ createZkNode(selectorParentPath);
+ // 插入或更新数据
+ insertZkNode(selectorRealPath, data);
+ }
+ }
+
+ // 创建 zk 节点
+ private void createZkNode(final String path) {
+ // 不存在才创建
+ if (!zkClient.exists(path)) {
+ zkClient.createPersistent(path, true);
+ }
+ }
+
+ // 插入zk节点
+ private void insertZkNode(final String path, final Object data) {
+ // 创建节点
+ createZkNode(path);
+ // 通过 zkClient 写入数据
+ zkClient.writeData(path, null == data ? "" : GsonUtils.getInstance().toJson(data));
+ }
+
+}
+```
+
+只要将变动的数据正确写入到`zk`的节点上,`admin`这边的操作就执行完成了。`ShenYu`在使用`zk`进行数据同步时,`zk`的节点是通过精心设计的。
+
+在我们当前的案例中,对`Divide`插件中的一条选择器数据进行更新,将权重更新为90,就会对图中的特定节点更新。
+
+
+
+
+我们用时序图将上面的更新流程串联起来。
+
+
+
+
+### 3. 网关数据同步
+
+假设`ShenYu`网关已经在正常运行,使用的数据同步方式也是`zookeeper`。那么当在`admin`端更新选择器数据后,并且向`zk`发送了变更的数据,那网关是如何接收并处理数据的呢?接下来我们就继续进行源码分析,一探究竟。
+
+#### 3.1 ZkClient接收数据
+
+- ZkClient.subscribeDataChanges()
+
+在网关端有一个`ZookeeperSyncDataService`类,它通过`ZkClient`订阅了数据节点,当数据发生变更时,可以感知到。
+
+```java
+/**
+ * 使用 zookeeper 缓存数据
+ */
+public class ZookeeperSyncDataService implements SyncDataService, AutoCloseable {
+
+private void subscribeSelectorDataChanges(final String path) {
+ // zkClient订阅数据节点
+ zkClient.subscribeDataChanges(path, new IZkDataListener() {
+ @Override
+ public void handleDataChange(final String dataPath, final Object data) {
+ cacheSelectorData(GsonUtils.getInstance().fromJson(data.toString(), SelectorData.class)); // 节点数据被更新
+ }
+
+ @Override
+ public void handleDataDeleted(final String dataPath) {
+ unCacheSelectorData(dataPath); // 节点数据被删除
+ }
+ });
+ }
+
+ // 省略了其他逻辑
+}
+```
+
+`ZooKeeper`的`Watch`机制,会给订阅的客户端发送节点变更通知。在我们的案例中,更新选择器信息,就会进入到`handleDataChange()`方法。通过`cacheSelectorData()`去处理数据。
+
+#### 3.2 处理数据
+
+- ZookeeperSyncDataService.cacheSelectorData()
+
+经过判空逻辑之后,缓存选择器数据的操作又交给了`PluginDataSubscriber`处理。
+
+```java
+ private void cacheSelectorData(final SelectorData selectorData) {
+ Optional.ofNullable(selectorData)
+ .ifPresent(data -> Optional.ofNullable(pluginDataSubscriber).ifPresent(e -> e.onSelectorSubscribe(data)));
+ }
+```
+
+`PluginDataSubscriber`是一个接口,它只有一个`CommonPluginDataSubscriber`实现类,负责处理插件、选择器和规则数据。
+
+
+#### 3.3 通用插件数据订阅者
+
+- PluginDataSubscriber.onSelectorSubscribe()
+
+它没有其他逻辑,直接调用`subscribeDataHandler()`方法。在方法中,更具数据类型(插件、选择器或规则),操作类型(更新或删除),去执行不同逻辑。
+
+```java
+/**
+ * 通用插件数据订阅者,负责处理所有插件、选择器和规则信息
+ * The type Common plugin data subscriber.
+ */
+public class CommonPluginDataSubscriber implements PluginDataSubscriber {
+ //......
+ // 处理选择器数据
+ @Override
+ public void onSelectorSubscribe(final SelectorData selectorData) {
+ subscribeDataHandler(selectorData, DataEventTypeEnum.UPDATE);
+ }
+
+ // 订阅数据处理器,处理数据的更新或删除
+ private <T> void subscribeDataHandler(final T classData, final DataEventTypeEnum dataType) {
+ Optional.ofNullable(classData).ifPresent(data -> {
+ // 插件数据
+ if (data instanceof PluginData) {
+ PluginData pluginData = (PluginData) data;
+ if (dataType == DataEventTypeEnum.UPDATE) { // 更新操作
+ // 将数据保存到网关内存
+ BaseDataCache.getInstance().cachePluginData(pluginData);
+ // 如果每个插件还有自己的处理逻辑,那么就去处理
+ Optional.ofNullable(handlerMap.get(pluginData.getName())).ifPresent(handler -> handler.handlerPlugin(pluginData));
+ } else if (dataType == DataEventTypeEnum.DELETE) { // 删除操作
+ // 从网关内存移除数据
+ BaseDataCache.getInstance().removePluginData(pluginData);
+ // 如果每个插件还有自己的处理逻辑,那么就去处理
+ Optional.ofNullable(handlerMap.get(pluginData.getName())).ifPresent(handler -> handler.removePlugin(pluginData));
+ }
+ } else if (data instanceof SelectorData) { // 选择器数据
+ SelectorData selectorData = (SelectorData) data;
+ if (dataType == DataEventTypeEnum.UPDATE) { // 更新操作
+ // 将数据保存到网关内存
+ BaseDataCache.getInstance().cacheSelectData(selectorData);
+ // 如果每个插件还有自己的处理逻辑,那么就去处理 Optional.ofNullable(handlerMap.get(selectorData.getPluginName())).ifPresent(handler -> handler.handlerSelector(selectorData));
+ } else if (dataType == DataEventTypeEnum.DELETE) { // 删除操作
+ // 从网关内存移除数据
+ BaseDataCache.getInstance().removeSelectData(selectorData);
+ // 如果每个插件还有自己的处理逻辑,那么就去处理
+ Optional.ofNullable(handlerMap.get(selectorData.getPluginName())).ifPresent(handler -> handler.removeSelector(selectorData));
+ }
+ } else if (data instanceof RuleData) { // 规则数据
+ RuleData ruleData = (RuleData) data;
+ if (dataType == DataEventTypeEnum.UPDATE) { // 更新操作
+ // 将数据保存到网关内存
+ BaseDataCache.getInstance().cacheRuleData(ruleData);
+ // 如果每个插件还有自己的处理逻辑,那么就去处理
+ Optional.ofNullable(handlerMap.get(ruleData.getPluginName())).ifPresent(handler -> handler.handlerRule(ruleData));
+ } else if (dataType == DataEventTypeEnum.DELETE) { // 删除操作
+ // 从网关内存移除数据
+ BaseDataCache.getInstance().removeRuleData(ruleData);
+ // 如果每个插件还有自己的处理逻辑,那么就去处理
+ Optional.ofNullable(handlerMap.get(ruleData.getPluginName())).ifPresent(handler -> handler.removeRule(ruleData));
+ }
+ }
+ });
+ }
+
+}
+```
+
+
+#### 3.4 数据缓存到内存
+
+那么更新一条选择器数据,会进入下面的逻辑:
+
+```java
+// 将数据保存到网关内存
+BaseDataCache.getInstance().cacheSelectData(selectorData);
+// 如果每个插件还有自己的处理逻辑,那么就去处理
+Optional.ofNullable(handlerMap.get(selectorData.getPluginName())).ifPresent(handler -> handler.handlerSelector(selectorData));
+```
+
+一是将数据保存到网关的内存中。`BaseDataCache`是最终缓存数据的类,通过单例模式实现。选择器数据就存到了`SELECTOR_MAP`这个`Map`中。在后续使用的时候,也是从这里拿数据。
+
+```java
+public final class BaseDataCache {
+ // 私有变量
+ private static final BaseDataCache INSTANCE = new BaseDataCache();
+ // 私有构造器
+ private BaseDataCache() {
+ }
+
+ /**
+ * Gets instance.
+ * 公开方法
+ * @return the instance
+ */
+ public static BaseDataCache getInstance() {
+ return INSTANCE;
+ }
+
+ /**
+ * 缓存选择器数据的Map
+ * pluginName -> SelectorData.
+ */
+ private static final ConcurrentMap<String, List<SelectorData>> SELECTOR_MAP = Maps.newConcurrentMap();
+
+ public void cacheSelectData(final SelectorData selectorData) {
+ Optional.ofNullable(selectorData).ifPresent(this::selectorAccept);
+ }
+
+ /**
+ * cache selector data.
+ * 缓存选择器数据
+ * @param data the selector data
+ */
+ private void selectorAccept(final SelectorData data) {
+ String key = data.getPluginName();
+ if (SELECTOR_MAP.containsKey(key)) { // 更新操作,先删除再插入
+ List<SelectorData> existList = SELECTOR_MAP.get(key);
+ final List<SelectorData> resultList = existList.stream().filter(r -> !r.getId().equals(data.getId())).collect(Collectors.toList());
+ resultList.add(data);
+ final List<SelectorData> collect = resultList.stream().sorted(Comparator.comparing(SelectorData::getSort)).collect(Collectors.toList());
+ SELECTOR_MAP.put(key, collect);
+ } else { // 新增操作,直接放到Map中
+ SELECTOR_MAP.put(key, Lists.newArrayList(data));
+ }
+ }
+
+}
+```
+
+二是如果每个插件还有自己的处理逻辑,那么就去处理。 通过`idea`编辑器可以看到,当新增一条选择器后,有如下的插件还有处理。这里我们就不再展开了。
+
+
+
+经过以上的源码追踪,并通过一个实际的案例,在`admin`端新增更新一条选择器数据,就将`zookeeper`数据同步的流程分析清楚了。
+
+我们还是通过时序图将网关端的数据同步流程串联一下:
+
+
+
+数据同步的流程已经分析完了,为了不让同步流程被打断,在分析过程中就忽略了其他逻辑。我们还需要分析`Admin`同步数据初始化和网关同步操作初始化的流程。
+
+### 4. Admin同步数据初始化
+
+当`admin`启动后,会将当前的数据信息全量同步到`zk`中,实现逻辑如下:
+
+```java
+
+/**
+ * Zookeeper 数据初始化
+ */
+public class ZookeeperDataInit implements CommandLineRunner {
+
+ private final ZkClient zkClient;
+
+ private final SyncDataService syncDataService;
+
+ /**
+ * Instantiates a new Zookeeper data init.
+ *
+ * @param zkClient the zk client
+ * @param syncDataService the sync data service
+ */
+ public ZookeeperDataInit(final ZkClient zkClient, final SyncDataService syncDataService) {
+ this.zkClient = zkClient;
+ this.syncDataService = syncDataService;
+ }
+
+ @Override
+ public void run(final String... args) {
+ String pluginPath = DefaultPathConstants.PLUGIN_PARENT;
+ String authPath = DefaultPathConstants.APP_AUTH_PARENT;
+ String metaDataPath = DefaultPathConstants.META_DATA;
+ // 判断zk中是否存在数据
+ if (!zkClient.exists(pluginPath) && !zkClient.exists(authPath) && !zkClient.exists(metaDataPath)) {
+ syncDataService.syncAll(DataEventTypeEnum.REFRESH);
+ }
+ }
+}
+
+```
+
+判断`zk`中是否存在数据,如果不存在,则进行同步。
+
+`ZookeeperDataInit`实现了`CommandLineRunner`接口。它是`springboot`提供的接口,会在所有 `Spring Beans`初始化之后执行`run()`方法,常用于项目中初始化的操作。
+
+- SyncDataService.syncAll()
+
+从数据库查询数据,然后进行全量数据同步,所有的认证信息、插件信息、选择器信息、规则信息和元数据信息。主要是通过`eventPublisher`发布同步事件。这里就跟前面提到的同步逻辑就又联系起来了,`eventPublisher`通过`publishEvent()`发布完事件后,有`ApplicationListener`执行事件变更操作,在`ShenYu`中就是前面提到的`DataChangedEventDispatcher`。
+
+```java
+@Service
+public class SyncDataServiceImpl implements SyncDataService {
+ // 事件发布
+ private final ApplicationEventPublisher eventPublisher;
+
+ /***
+ * 全量数据同步
+ * @param type the type
+ * @return
+ */
+ @Override
+ public boolean syncAll(final DataEventTypeEnum type) {
+ // 同步认证信息
+ appAuthService.syncData();
+ // 同步插件信息
+ List<PluginData> pluginDataList = pluginService.listAll();
+ eventPublisher.publishEvent(new DataChangedEvent(ConfigGroupEnum.PLUGIN, type, pluginDataList));
+ // 同步选择器信息
+ List<SelectorData> selectorDataList = selectorService.listAll();
+ eventPublisher.publishEvent(new DataChangedEvent(ConfigGroupEnum.SELECTOR, type, selectorDataList));
+ // 同步规则信息
+ List<RuleData> ruleDataList = ruleService.listAll();
+ eventPublisher.publishEvent(new DataChangedEvent(ConfigGroupEnum.RULE, type, ruleDataList));
+ // 同步元数据信息
+ metaDataService.syncData();
+ return true;
+ }
+
+}
+```
+
+### 5. 网关同步操作初始化
+
+网关这边的数据同步初始化操作主要是订阅`zk`中的节点,当有数据变更时,收到变更数据。这依赖于`ZooKeeper`的`Watch`机制。在`ShenYu`中,负责`zk`数据同步的是`ZookeeperSyncDataService`,也在前面提到过。
+
+
+`ZookeeperSyncDataService`的功能逻辑是在实例化的过程中完成的:对`zk`中的`shenyu`数据同步节点完成订阅。这里的订阅分两类,一类是已经存在的节点上面数据发生更新,这通过`zkClient.subscribeDataChanges()`方法实现;另一类是当前节点下有新增或删除节点,即子节点发生变化,这通过`zkClient.subscribeChildChanges()`方法实现。
+
+
+`ZookeeperSyncDataService`的代码有点多,这里我们以插件数据的读取和订阅进行追踪,其他类型的数据操作原理是一样的。
+
+```java
+
+/**
+ * zookeeper 数据同步服务
+ */
+public class ZookeeperSyncDataService implements SyncDataService, AutoCloseable {
+ // 在实例化的时候,完成从zk中读取数据的操作,并订阅节点
+ public ZookeeperSyncDataService( /*省略构造参数参数*/ ) {
+ this.zkClient = zkClient;
+ this.pluginDataSubscriber = pluginDataSubscriber;
+ this.metaDataSubscribers = metaDataSubscribers;
+ this.authDataSubscribers = authDataSubscribers;
+ // 订阅插件、选择器和规则数据
+ watcherData();
+ // 订阅认证数据
+ watchAppAuth();
+ // 订阅元数据
+ watchMetaData();
+ }
+
+ private void watcherData() {
+ // 插件节点路径
+ final String pluginParent = DefaultPathConstants.PLUGIN_PARENT;
+ // 所有插件节点
+ List<String> pluginZKs = zkClientGetChildren(pluginParent);
+ for (String pluginName : pluginZKs) {
+ // 订阅当前所有插件、选择器和规则数据
+ watcherAll(pluginName);
+ }
+ // 订阅子节点(新增或删除一个插件)
+ zkClient.subscribeChildChanges(pluginParent, (parentPath, currentChildren) -> {
+ if (CollectionUtils.isNotEmpty(currentChildren)) {
+ for (String pluginName : currentChildren) {
+ // 需要订阅子节点的所有插件、选择器和规则数据
+ watcherAll(pluginName);
+ }
+ }
+ });
+ }
+
+ private void watcherAll(final String pluginName) {
+ // 订阅插件数据
+ watcherPlugin(pluginName);
+ // 订阅选择器数据
+ watcherSelector(pluginName);
+ // 订阅规则数据
+ watcherRule(pluginName);
+ }
+
+ private void watcherPlugin(final String pluginName) {
+ // 当前插件路径
+ String pluginPath = DefaultPathConstants.buildPluginPath(pluginName);
+ // 是否存在,不存在就创建
+ if (!zkClient.exists(pluginPath)) {
+ zkClient.createPersistent(pluginPath, true);
+ }
+ // 读取zk上当前节点数据,并反序列化
+ PluginData pluginData = null == zkClient.readData(pluginPath) ? null
+ : GsonUtils.getInstance().fromJson((String) zkClient.readData(pluginPath), PluginData.class);
+ // 缓存到网关内存中
+ cachePluginData(pluginData);
+ // 订阅插件节点
+ subscribePluginDataChanges(pluginPath, pluginName);
+ }
+
+ private void cachePluginData(final PluginData pluginData) {
+ // 省略实现逻辑,其实就是 CommonPluginDataSubscriber 中的操作,跟前面都能联系起来
+ }
+
+ private void subscribePluginDataChanges(final String pluginPath, final String pluginName) {
+ // 订阅数据变更:更新或删除
+ zkClient.subscribeDataChanges(pluginPath, new IZkDataListener() {
+
+ @Override
+ public void handleDataChange(final String dataPath, final Object data) { // 更新操作
+ // 省略实现逻辑,其实就是 CommonPluginDataSubscriber 中的操作,跟前面都能联系起来
+ }
+
+ @Override
+ public void handleDataDeleted(final String dataPath) { // 删除操作
+ // 省略实现逻辑,其实就是 CommonPluginDataSubscriber 中的操作,跟前面都能联系起来
+
+ }
+ });
+ }
+
+}
+```
+
+上面的源代码中都给出了注释,相信大家可以看明白。订阅插件数据的主要逻辑如下:
+
+> 1. 构造当前插件路径
+> 2. 路径是否存在,不存在就创建
+> 3. 读取zk上当前节点数据,并反序列化
+> 4. 插件数据缓存到网关内存中
+> 5. 订阅插件节点
+
+
+### 6. 总结
+
+本文通过一个实际案例,对`zookeeper`的数据同步原理进行了源码分析。涉及到的主要知识点如下:
+
+- 基于`zookeeper`的数据同步,主要是通过`watch`机制实现;
+- 通过`Spring`完成事件发布和监听;
+- 通过抽象`DataChangedListener`接口,支持多种同步策略,面向接口编程;
+- 使用单例设计模式实现缓存数据类`BaseDataCache`;
+- 通过`SpringBoot`的条件装配和`starter`加载机制实现配置类的加载。
diff --git a/static/img/activities/code-analysis-zookeeper-data-sync/data-changed-listener.png b/static/img/activities/code-analysis-zookeeper-data-sync/data-changed-listener.png
new file mode 100644
index 0000000..6a0e9d2
Binary files /dev/null and b/static/img/activities/code-analysis-zookeeper-data-sync/data-changed-listener.png differ
diff --git a/static/img/activities/code-analysis-zookeeper-data-sync/handler-selector.png b/static/img/activities/code-analysis-zookeeper-data-sync/handler-selector.png
new file mode 100644
index 0000000..2ff9d2b
Binary files /dev/null and b/static/img/activities/code-analysis-zookeeper-data-sync/handler-selector.png differ
diff --git a/static/img/activities/code-analysis-zookeeper-data-sync/update-selector-en.png b/static/img/activities/code-analysis-zookeeper-data-sync/update-selector-en.png
new file mode 100644
index 0000000..76a6a91
Binary files /dev/null and b/static/img/activities/code-analysis-zookeeper-data-sync/update-selector-en.png differ
diff --git a/static/img/activities/code-analysis-zookeeper-data-sync/update-selector-zh.png b/static/img/activities/code-analysis-zookeeper-data-sync/update-selector-zh.png
new file mode 100644
index 0000000..00c9559
Binary files /dev/null and b/static/img/activities/code-analysis-zookeeper-data-sync/update-selector-zh.png differ
diff --git a/static/img/activities/code-analysis-zookeeper-data-sync/zk-sync-sequence-admin-en.png b/static/img/activities/code-analysis-zookeeper-data-sync/zk-sync-sequence-admin-en.png
new file mode 100644
index 0000000..4f582e2
Binary files /dev/null and b/static/img/activities/code-analysis-zookeeper-data-sync/zk-sync-sequence-admin-en.png differ
diff --git a/static/img/activities/code-analysis-zookeeper-data-sync/zk-sync-sequence-admin-zh.png b/static/img/activities/code-analysis-zookeeper-data-sync/zk-sync-sequence-admin-zh.png
new file mode 100644
index 0000000..01de6de
Binary files /dev/null and b/static/img/activities/code-analysis-zookeeper-data-sync/zk-sync-sequence-admin-zh.png differ
diff --git a/static/img/activities/code-analysis-zookeeper-data-sync/zk-sync-sequence-gateway-en.png b/static/img/activities/code-analysis-zookeeper-data-sync/zk-sync-sequence-gateway-en.png
new file mode 100644
index 0000000..9bb05bd
Binary files /dev/null and b/static/img/activities/code-analysis-zookeeper-data-sync/zk-sync-sequence-gateway-en.png differ
diff --git a/static/img/activities/code-analysis-zookeeper-data-sync/zk-sync-sequence-gateway-zh.png b/static/img/activities/code-analysis-zookeeper-data-sync/zk-sync-sequence-gateway-zh.png
new file mode 100644
index 0000000..dd90ace
Binary files /dev/null and b/static/img/activities/code-analysis-zookeeper-data-sync/zk-sync-sequence-gateway-zh.png differ
diff --git a/static/img/activities/code-analysis-zookeeper-data-sync/zookeeper-node.png b/static/img/activities/code-analysis-zookeeper-data-sync/zookeeper-node.png
new file mode 100644
index 0000000..31cb75b
Binary files /dev/null and b/static/img/activities/code-analysis-zookeeper-data-sync/zookeeper-node.png differ