You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by zh...@apache.org on 2021/12/17 06:25:36 UTC
[dolphinscheduler-website] branch master updated: Add cache doc (#590)
This is an automated email from the ASF dual-hosted git repository.
zhongjiajie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler-website.git
The following commit(s) were added to refs/heads/master by this push:
new cc01566 Add cache doc (#590)
cc01566 is described below
commit cc01566f164fd653c6fd70281ffe61c06f7ecc75
Author: wind <ca...@users.noreply.github.com>
AuthorDate: Fri Dec 17 14:25:32 2021 +0800
Add cache doc (#590)
* add cache doc
---
docs/en-us/dev/user_doc/architecture/cache.md | 42 ++++++++++++++++++++++++++
docs/zh-cn/dev/user_doc/architecture/cache.md | 42 ++++++++++++++++++++++++++
img/cache-evict.png | Bin 0 -> 56216 bytes
site_config/docsdev.js | 8 +++++
4 files changed, 92 insertions(+)
diff --git a/docs/en-us/dev/user_doc/architecture/cache.md b/docs/en-us/dev/user_doc/architecture/cache.md
new file mode 100644
index 0000000..6a7359d
--- /dev/null
+++ b/docs/en-us/dev/user_doc/architecture/cache.md
@@ -0,0 +1,42 @@
+### Cache
+
+#### Purpose
+
+Due to the master-server scheduling process, there will be a large number of database read operations, such as `tenant`, `user`, `processDefinition`, etc. On the one hand, it will put a lot of pressure on the DB, and on the other hand, it will slow down the entire core scheduling process.
+
+Considering that this part of the business data is a scenario where more reads and less writes are performed, a cache module is introduced to reduce the DB read pressure and speed up the core scheduling process;
+
+#### Cache settings
+
+```yaml
+spring:
+ cache:
+ # default disable cache, you can enable by `type: caffeine`
+ type: none
+ cache-names:
+ - tenant
+ - user
+ - processDefinition
+ - processTaskRelation
+ - taskDefinition
+ caffeine:
+ spec: maximumSize=100,expireAfterWrite=300s,recordStats
+```
+
+The cache-module use [spring-cache](https://spring.io/guides/gs/caching/), so you can set cache config in the spring application.yaml directly. Default disable cache, and you can enable it by `type: caffeine`.
+
+With the config of [caffeine](https://github.com/ben-manes/caffeine), you can set the cache size, expire time, etc.
+
+#### Cache Read
+
+The cache adopts the annotation `@Cacheable` of spring-cache and is configured in the mapper layer. For example: `TenantMapper`.
+
+#### Cache Evict
+
+The business data update comes from the api-server, and the cache end is in the master-server. So it is necessary to monitor the data update of the api-server (aspect intercept `@CacheEvict`), and the master-server will be notified when the cache eviction is required.
+
+It should be noted that the final strategy for cache update comes from the user's expiration strategy configuration in caffeine, so please configure it in conjunction with the business;
+
+The sequence diagram is shown in the following figure:
+
+<img src="/img/cache-evict.png" alt="cache-evict" style="zoom: 67%;" />
\ No newline at end of file
diff --git a/docs/zh-cn/dev/user_doc/architecture/cache.md b/docs/zh-cn/dev/user_doc/architecture/cache.md
new file mode 100644
index 0000000..8b4804d
--- /dev/null
+++ b/docs/zh-cn/dev/user_doc/architecture/cache.md
@@ -0,0 +1,42 @@
+### 缓存
+
+#### 缓存目的
+
+由于在master-server调度过程中，会产生大量的数据库读取操作，如tenant，user，processDefinition等，一方面对DB产生很大的读压力，另一方面则会使整个核心调度流程变得缓慢；
+
+考虑到这部分业务数据是读多写少的场景，故引入了缓存模块，以减少DB读压力，加快核心调度流程；
+
+#### 缓存设置
+
+```yaml
+spring:
+ cache:
+ # default enable cache, you can disable by `type: none`
+ type: none
+ cache-names:
+ - tenant
+ - user
+ - processDefinition
+ - processTaskRelation
+ - taskDefinition
+ caffeine:
+ spec: maximumSize=100,expireAfterWrite=300s,recordStats
+```
+
+缓存模块采用[spring-cache](https://spring.io/guides/gs/caching/)机制，可直接在spring配置文件中配置是否开启缓存（默认`none`关闭）, 缓存类型；
+
+目前采用[caffeine](https://github.com/ben-manes/caffeine)进行缓存管理，可自由设置缓存相关配置，如缓存大小、过期时间等；
+
+#### 缓存读取
+
+缓存采用spring-cache的注解，配置在相关的mapper层，可参考如：`TenantMapper`.
+
+#### 缓存更新
+
+业务数据的更新来自于api-server, 而缓存端在master-server, 故需要对api-server的数据更新做监听(aspect切面拦截`@CacheEvict`)，当需要进行缓存驱逐时会通知master-server，master-server接收到cacheEvictCommand后进行缓存驱逐；
+
+需要注意的是：缓存更新的兜底策略来自于用户在caffeine中的过期策略配置，请结合业务进行配置；
+
+时序图如下图所示：
+
+<img src="/img/cache-evict.png" alt="cache-evict" style="zoom: 67%;" />
\ No newline at end of file
diff --git a/img/cache-evict.png b/img/cache-evict.png
new file mode 100644
index 0000000..4cb5d13
Binary files /dev/null and b/img/cache-evict.png differ
diff --git a/site_config/docsdev.js b/site_config/docsdev.js
index 57ffc40..2e882d0 100644
--- a/site_config/docsdev.js
+++ b/site_config/docsdev.js
@@ -222,6 +222,10 @@ export default {
title: 'Load Balance',
link: '/en-us/docs/dev/user_doc/architecture/load-balance.html',
},
+ {
+ title: 'Cache',
+ link: '/en-us/docs/dev/user_doc/architecture/cache.html',
+ },
],
},
{
@@ -495,6 +499,10 @@ export default {
title: '负载均衡',
link: '/zh-cn/docs/dev/user_doc/architecture/load-balance.html',
},
+ {
+ title: '缓存',
+ link: '/zh-cn/docs/dev/user_doc/architecture/cache.html',
+ },
],
},
{