You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by du...@apache.org on 2023/02/01 02:48:53 UTC
[shardingsphere] branch master updated: Implements GENERIC_TABLE_RANDOM_REPLACE mask algorithm (#23685)
This is an automated email from the ASF dual-hosted git repository.
duanzhengqiang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/shardingsphere.git
The following commit(s) were added to refs/heads/master by this push:
new 23977f38073 Implements GENERIC_TABLE_RANDOM_REPLACE mask algorithm (#23685)
23977f38073 is described below
commit 23977f380735d769389f7fc9d0f451587fbf2564
Author: lushaorong <ww...@163.com>
AuthorDate: Wed Feb 1 10:48:46 2023 +0800
Implements GENERIC_TABLE_RANDOM_REPLACE mask algorithm (#23685)
* Implements GENERIC_TABLE_RANDOM_REPLACE mask algorithm
Signed-off-by: lushaorong <ww...@163.com>
* update mask algorithm document
Signed-off-by: lushaorong <ww...@163.com>
* fix resources
Signed-off-by: lushaorong <ww...@163.com>
* add default generic tables
Signed-off-by: lushaorong <ww...@163.com>
---------
Signed-off-by: lushaorong <ww...@163.com>
---
docs/document/content/dev-manual/mask.cn.md | 1 +
docs/document/content/dev-manual/mask.en.md | 1 +
.../common-config/builtin-algorithm/mask.cn.md | 13 +++
.../common-config/builtin-algorithm/mask.en.md | 13 +++
.../GenericTableRandomReplaceAlgorithm.java | 106 +++++++++++++++++++++
...rg.apache.shardingsphere.mask.spi.MaskAlgorithm | 1 +
.../GenericTableRandomReplaceAlgorithmTest.java | 62 ++++++++++++
7 files changed, 197 insertions(+)
diff --git a/docs/document/content/dev-manual/mask.cn.md b/docs/document/content/dev-manual/mask.cn.md
index 4e33f7f11b3..3273414376e 100644
--- a/docs/document/content/dev-manual/mask.cn.md
+++ b/docs/document/content/dev-manual/mask.cn.md
@@ -30,3 +30,4 @@ chapter = true
| MILITARY_IDENTITY_NUMBER_RANDOM_REPLACE | 军官证随机替换数据脱敏算法 | [`org.apache.shardingsphere.mask.algorithm.replace.MilitaryIdentityNumberRandomReplaceAlgorithm`](https://github.com/apache/shardingsphere/blob/master/features/mask/core/src/main/java/org/apache/shardingsphere/mask/algorithm/replace/MilitaryIdentityNumberRandomReplaceAlgorithm.java) |
| TELEPHONE_RANDOM_REPLACE | ⼿机号随机替换数据脱敏算法 | [`org.apache.shardingsphere.mask.algorithm.replace.TelephoneRandomReplaceAlgorithm`](https://github.com/apache/shardingsphere/blob/master/features/mask/core/src/main/java/org/apache/shardingsphere/mask/algorithm/replace/TelephoneRandomReplaceAlgorithm.java) |
| LANDLINE_NUMBER_RANDOM_REPLACE | 座机号码随机替换 | [`org.apache.shardingsphere.mask.algorithm.replace.LandlineNumberRandomAlgorithm`](https://github.com/apache/shardingsphere/blob/master/features/mask/core/src/main/java/org/apache/shardingsphere/mask/algorithm/replace/LandlineNumberRandomAlgorithm.java) |
+| GENERIC_TABLE_RANDOM_REPLACE | 通⽤表格随机替换 | [`org.apache.shardingsphere.mask.algorithm.replace.GenericTableRandomReplaceAlgorithm`](https://github.com/apache/shardingsphere/blob/master/features/mask/core/src/main/java/org/apache/shardingsphere/mask/algorithm/replace/GenericTableRandomReplaceAlgorithm.java) |
diff --git a/docs/document/content/dev-manual/mask.en.md b/docs/document/content/dev-manual/mask.en.md
index 09f56489e9b..025712c3fb8 100644
--- a/docs/document/content/dev-manual/mask.en.md
+++ b/docs/document/content/dev-manual/mask.en.md
@@ -30,3 +30,4 @@ Data masking algorithm definition
| MILITARY_IDENTITY_NUMBER_RANDOM_REPLACE | Military identity number random replace data masking algorithm | [`org.apache.shardingsphere.mask.algorithm.replace.MilitaryIdentityNumberRandomReplaceAlgorithm`](https://github.com/apache/shardingsphere/blob/master/features/mask/core/src/main/java/org/apache/shardingsphere/mask/algorithm/replace/MilitaryIdentityNumberRandomReplaceAlgorithm.java) |
| TELEPHONE_RANDOM_REPLACE | Telephone random replace data masking algorithm | [`org.apache.shardingsphere.mask.algorithm.replace.TelephoneRandomReplaceAlgorithm`](https://github.com/apache/shardingsphere/blob/master/features/mask/core/src/main/java/org/apache/shardingsphere/mask/algorithm/replace/TelephoneRandomReplaceAlgorithm.java) |
| LANDLINE_NUMBER_RANDOM_REPLACE | Landline number random replace data masking algorithm | [`org.apache.shardingsphere.mask.algorithm.replace.LandlineNumberRandomAlgorithm`](https://github.com/apache/shardingsphere/blob/master/features/mask/core/src/main/java/org/apache/shardingsphere/mask/algorithm/replace/LandlineNumberRandomAlgorithm.java) |
+| GENERIC_TABLE_RANDOM_REPLACE | Generic table random replace algorithm | [`org.apache.shardingsphere.mask.algorithm.replace.GenericTableRandomReplaceAlgorithm`](https://github.com/apache/shardingsphere/blob/master/features/mask/core/src/main/java/org/apache/shardingsphere/mask/algorithm/replace/GenericTableRandomReplaceAlgorithm.java) |
diff --git a/docs/document/content/user-manual/common-config/builtin-algorithm/mask.cn.md b/docs/document/content/user-manual/common-config/builtin-algorithm/mask.cn.md
index f2bcf75c297..2154998d8e6 100644
--- a/docs/document/content/user-manual/common-config/builtin-algorithm/mask.cn.md
+++ b/docs/document/content/user-manual/common-config/builtin-algorithm/mask.cn.md
@@ -135,6 +135,19 @@ weight = 9
|-----------------|----------|-------------|
| landline-numbers | String | 座机号码（以英文逗号分隔） |
+#### 通⽤表格随机替换
+
+类型：GENERIC_TABLE_RANDOM_REPLACE
+
+可配置属性：
+
+| *名称* | *数据类型* | *说明* |
+|-----------------|----------|-------------|
+| uppercase-letter-codes | String | 大写字母 |
+| lowercase-letter-codes | String | 小写字母 |
+| digital-random-codes | String | 数字 |
+| special-codes | String | 特殊字符 |
+
## 操作步骤
1. 在脱敏规则中配置脱敏算法；
2. 为脱敏算法指定脱敏算法类型。
diff --git a/docs/document/content/user-manual/common-config/builtin-algorithm/mask.en.md b/docs/document/content/user-manual/common-config/builtin-algorithm/mask.en.md
index d92808032cc..d442057d267 100644
--- a/docs/document/content/user-manual/common-config/builtin-algorithm/mask.en.md
+++ b/docs/document/content/user-manual/common-config/builtin-algorithm/mask.en.md
@@ -135,6 +135,19 @@ Attributes:
|-----------------|--------|---------------------------------------|
| landline-numbers | String | Landline numbers (separate with comma) |
+#### Generic table random replace algorithm.
+
+Type: GENERIC_TABLE_RANDOM_REPLACE
+
+Attributes:
+
+| *Name* | *DataType* | *Description* |
+|-----------------|--------|---------------------------------------|
+| uppercase-letter-codes | String | Uppercase letter (No Split) |
+| lowercase-letter-codes | String | Lowercase-letter (No Split) |
+| digital-random-codes | String | Number (No Split) |
+| special-codes | String | Special code (No Split)|
+
## Operating Procedure
1. Configure maskAlgorithms in a mask rule.
2. Use relevant algorithm types in maskAlgorithms.
diff --git a/features/mask/core/src/main/java/org/apache/shardingsphere/mask/algorithm/replace/GenericTableRandomReplaceAlgorithm.java b/features/mask/core/src/main/java/org/apache/shardingsphere/mask/algorithm/replace/GenericTableRandomReplaceAlgorithm.java
new file mode 100644
index 00000000000..d6af6796ff6
--- /dev/null
+++ b/features/mask/core/src/main/java/org/apache/shardingsphere/mask/algorithm/replace/GenericTableRandomReplaceAlgorithm.java
@@ -0,0 +1,106 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shardingsphere.mask.algorithm.replace;
+
+import com.google.common.base.Strings;
+import lombok.Getter;
+import org.apache.shardingsphere.mask.spi.MaskAlgorithm;
+
+import java.util.Properties;
+import java.util.Random;
+
+/**
+ * Generic table random replace algorithm.
+ */
+public final class GenericTableRandomReplaceAlgorithm implements MaskAlgorithm<Object, String> {
+
+ private static final String UPPERCASE_LETTER_CODES = "uppercase-letter-codes";
+
+ private static final String LOWERCASE_LETTER_CODES = "lowercase-letter-codes";
+
+ private static final String DIGITAL_RANDOM_CODES = "digital-random-codes";
+
+ private static final String SPECIAL_CODES = "special-codes";
+
+ private String uppercaseLetterCodes = "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
+
+ private String lowercaseLetterCodes = "abcdefghijklmnopqrstuvwxyz";
+
+ private String digitalRandomCodes = "0123456789";
+
+ private String specialCodes = "~!@#$%^&*:<>|";
+
+ @Getter
+ private Properties props;
+
+ @Override
+ public void init(final Properties props) {
+ this.props = props;
+
+ if (isLeastOneCharConfig(props, UPPERCASE_LETTER_CODES)) {
+ this.uppercaseLetterCodes = props.getProperty(UPPERCASE_LETTER_CODES);
+ }
+ if (isLeastOneCharConfig(props, LOWERCASE_LETTER_CODES)) {
+ this.lowercaseLetterCodes = props.getProperty(LOWERCASE_LETTER_CODES);
+ }
+ if (isLeastOneCharConfig(props, DIGITAL_RANDOM_CODES)) {
+ this.digitalRandomCodes = props.getProperty(DIGITAL_RANDOM_CODES);
+ }
+ if (isLeastOneCharConfig(props, SPECIAL_CODES)) {
+ this.specialCodes = props.getProperty(SPECIAL_CODES);
+ }
+ }
+
+ private boolean isLeastOneCharConfig(final Properties props, final String atLeastOneCharConfigKey) {
+ if (!props.containsKey(atLeastOneCharConfigKey)) {
+ return false;
+ }
+ if (0 == props.getProperty(atLeastOneCharConfigKey).length()) {
+ return false;
+ }
+ return true;
+ }
+
+ @Override
+ public String mask(final Object plainValue) {
+ String result = null == plainValue ? null : String.valueOf(plainValue);
+ if (Strings.isNullOrEmpty(result)) {
+ return result;
+ }
+ Random random = new Random();
+ char[] chars = result.toCharArray();
+ for (int i = 0; i < chars.length; i++) {
+ char c = chars[i];
+ if ('A' <= c && c <= 'Z') {
+ chars[i] = uppercaseLetterCodes.charAt(random.nextInt(uppercaseLetterCodes.length()));
+ } else if ('a' <= c && c <= 'z') {
+ chars[i] = lowercaseLetterCodes.charAt(random.nextInt(lowercaseLetterCodes.length()));
+ } else if ('0' <= c && c <= '9') {
+ chars[i] = digitalRandomCodes.charAt(random.nextInt(digitalRandomCodes.length()));
+ } else {
+ chars[i] = specialCodes.charAt(random.nextInt(specialCodes.length()));
+ }
+ }
+ return new String(chars);
+ }
+
+ @Override
+ public String getType() {
+ return "GENERIC_TABLE_RANDOM_REPLACE";
+ }
+}
diff --git a/features/mask/core/src/main/resources/META-INF/services/org.apache.shardingsphere.mask.spi.MaskAlgorithm b/features/mask/core/src/main/resources/META-INF/services/org.apache.shardingsphere.mask.spi.MaskAlgorithm
index 6984bea317d..fa34cf4c2fd 100644
--- a/features/mask/core/src/main/resources/META-INF/services/org.apache.shardingsphere.mask.spi.MaskAlgorithm
+++ b/features/mask/core/src/main/resources/META-INF/services/org.apache.shardingsphere.mask.spi.MaskAlgorithm
@@ -26,3 +26,4 @@ org.apache.shardingsphere.mask.algorithm.replace.TelephoneRandomReplaceAlgorithm
org.apache.shardingsphere.mask.algorithm.replace.PersonalIdentityNumberRandomReplaceAlgorithm
org.apache.shardingsphere.mask.algorithm.replace.MilitaryIdentityNumberRandomReplaceAlgorithm
org.apache.shardingsphere.mask.algorithm.replace.LandlineNumberRandomAlgorithm
+org.apache.shardingsphere.mask.algorithm.replace.GenericTableRandomReplaceAlgorithm
diff --git a/features/mask/core/src/test/java/org/apache/shardingsphere/mask/algorithm/replace/GenericTableRandomReplaceAlgorithmTest.java b/features/mask/core/src/test/java/org/apache/shardingsphere/mask/algorithm/replace/GenericTableRandomReplaceAlgorithmTest.java
new file mode 100644
index 00000000000..d6d119dee74
--- /dev/null
+++ b/features/mask/core/src/test/java/org/apache/shardingsphere/mask/algorithm/replace/GenericTableRandomReplaceAlgorithmTest.java
@@ -0,0 +1,62 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shardingsphere.mask.algorithm.replace;
+
+import org.apache.shardingsphere.test.util.PropertiesBuilder;
+import org.apache.shardingsphere.test.util.PropertiesBuilder.Property;
+import org.hamcrest.CoreMatchers;
+import org.junit.Before;
+import org.junit.Test;
+
+import java.util.Arrays;
+import java.util.Properties;
+import java.util.stream.Collectors;
+
+import static org.hamcrest.CoreMatchers.anyOf;
+import static org.hamcrest.CoreMatchers.is;
+import static org.hamcrest.MatcherAssert.assertThat;
+
+public final class GenericTableRandomReplaceAlgorithmTest {
+
+ private GenericTableRandomReplaceAlgorithm maskAlgorithm;
+
+ @Before
+ public void setUp() {
+ this.maskAlgorithm = new GenericTableRandomReplaceAlgorithm();
+ }
+
+ @Test
+ public void assertMask() {
+ maskAlgorithm.init(PropertiesBuilder.build(new Property("uppercase-letter-codes", "ABCD"), new Property("lowercase-letter-codes", "abcd"), new Property("digital-random-codes", "1234"),
+ new Property("special-codes", "~!@#")));
+ assertThat(maskAlgorithm.mask(""), is(""));
+ assertThat(maskAlgorithm.mask("Ab1!").charAt(0), anyOf(is('A'), is('B'), is('C'), is('D')));
+ assertThat(maskAlgorithm.mask("Ab1!").charAt(1), anyOf(is('a'), is('b'), is('c'), is('d')));
+ assertThat(maskAlgorithm.mask("Ab1!").charAt(2), anyOf(is('1'), is('2'), is('3'), is('4')));
+ assertThat(maskAlgorithm.mask("Ab1!").charAt(3), anyOf(is('~'), is('!'), is('@'), is('#')));
+ }
+
+ @Test
+ public void assertInitWithEmptyProps() {
+ maskAlgorithm.init(new Properties());
+ assertThat(maskAlgorithm.mask("Ab1!").substring(0, 1), anyOf(Arrays.stream("ABCDEFGHIJKLMNOPQRSTUVWXYZ".split("")).map(CoreMatchers::is).collect(Collectors.toList())));
+ assertThat(maskAlgorithm.mask("Ab1!").substring(1, 2), anyOf(Arrays.stream("abcdefghijklmnopqrstuvwxyz".split("")).map(CoreMatchers::is).collect(Collectors.toList())));
+ assertThat(maskAlgorithm.mask("Ab1!").substring(2, 3), anyOf(Arrays.stream("0123456789".split("")).map(CoreMatchers::is).collect(Collectors.toList())));
+ assertThat(maskAlgorithm.mask("Ab1!").substring(3, 4), anyOf(Arrays.stream("~!@#$%^&*:<>|".split("")).map(CoreMatchers::is).collect(Collectors.toList())));
+ }
+}