[GitHub] [kyuubi] Jackhjf commented on a diff in pull request #4531: [KYUUBI#4530] fix data masking does not support Chinese for strategies such as MASK_SHOW_FIRST_4 or MASK_SHOW_FIRST_4

["Jackhjf (via GitHub)" <gi...@apache.org>]

Jackhjf commented on code in PR #4531:
URL: https://github.com/apache/kyuubi/pull/4531#discussion_r1138302079


##########
extensions/spark/kyuubi-spark-authz/src/main/scala/org/apache/kyuubi/plugin/spark/authz/ranger/SparkRangerAdminPlugin.scala:
##########
@@ -135,7 +135,8 @@ object SparkRangerAdminPlugin extends RangerBasePlugin("spark", "sparkSql")
     val pos = if (hasLen) ", 5" else ""
     val upper = s"regexp_replace($expr, '[A-Z]', 'X'$pos)"
     val lower = s"regexp_replace($upper, '[a-z]', 'x'$pos)"
-    val digits = s"regexp_replace($lower, '[0-9]', 'n'$pos)"
+    val chinese = s"regexp_replace($lower, '[\u4e00-\u9fa5]', 'x'$pos)"

Review Comment:
   > What's the behavior on other popular databases/OLAP systems? And I remember that UTF-8 reserved range for CJK, can we extend it to cover CJK?
   Spark needs to access the metastore to query data, and the metadata information in the metastore is stored in mysql.In mysql, the character set is also utf-8
   <img width="594" alt="image" src="https://user-images.githubusercontent.com/33471639/225561693-42487689-dad6-4560-a563-ffcb8a0985ec.png">
   
   CJK can be covered by the following regular expression
   <img width="1435" alt="image" src="https://user-images.githubusercontent.com/33471639/225562034-4ee77e2d-1dd1-4777-b182-2add3e39aa85.png">
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org