You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2020/06/29 03:48:51 UTC
[GitHub] [shardingsphere] pengweizhong opened a new issue #6206: SpringBoot-YML configuration, data desensitization and database and table division strategies are not compatible
pengweizhong opened a new issue #6206:
URL: https://github.com/apache/shardingsphere/issues/6206
As mentioned in the title, when the data desensitization is configured in Spring Boot-YML, the sub-database and sub-table rules become invalid, causing the data to report "Table'xxx.xxx' doesn't exist." when it is inserted. After the data desensitization configuration is commented out, the sub-database and sub-table strategy takes effect. But I still can't find the problem.
I hope that the three functions of data desensitization, read-write separation, and database and table division can be used at the same time, thank you.
### Program environment
Operating system: win10
jdk version: 1.8
### pom dependency
```pom
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>sharding-jdbc-spring-boot-starter</artifactId>
<version>4.1.0</version>
</dependency>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>sharding-transaction-spring</artifactId>
<version>4.1.0</version>
</dependency>
```
### My YML configuration is listed below:
```yml
spring:
application:
name: application_service
shardingsphere:
datasource:
names: ds0,ds1,ds2,s0,s1,s2
# master DB
ds0:
type: com.zaxxer.hikari.HikariDataSource
driver-class-name: com.mysql.cj.jdbc.Driver
jdbcUrl: jdbc:mysql://localhost:3306/bank1?serverTimezone=UTC&characterEncoding=utf8&characterSetResults=utf8&autoReconnect=true&failOverReadOnly=false
username: root
password: pengwz
ds1:
type: com.zaxxer.hikari.HikariDataSource
driver-class-name: com.mysql.cj.jdbc.Driver
jdbcUrl: jdbc:mysql://localhost:3306/bank2?serverTimezone=UTC&characterEncoding=utf8&characterSetResults=utf8&autoReconnect=true&failOverReadOnly=false
username: root
password: pengwz
ds2:
type: com.zaxxer.hikari.HikariDataSource
driver-class-name: com.mysql.cj.jdbc.Driver
jdbcUrl: jdbc:mysql://localhost:3306/user?serverTimezone=UTC&characterEncoding=utf8&characterSetResults=utf8&autoReconnect=true&failOverReadOnly=false
username: root
password: pengwz
# slave DB
s0:
type: com.zaxxer.hikari.HikariDataSource
driver-class-name: com.mysql.cj.jdbc.Driver
jdbcUrl: jdbc:mysql://localhost:3307/bank1?serverTimezone=UTC&characterEncoding=utf8&characterSetResults=utf8&autoReconnect=true&failOverReadOnly=false
username: root
password: pengwz
s1:
type: com.zaxxer.hikari.HikariDataSource
driver-class-name: com.mysql.cj.jdbc.Driver
jdbcUrl: jdbc:mysql://localhost:3307/bank2?serverTimezone=UTC&characterEncoding=utf8&characterSetResults=utf8&autoReconnect=true&failOverReadOnly=false
username: root
password: pengwz
s2:
type: com.zaxxer.hikari.HikariDataSource
driver-class-name: com.mysql.cj.jdbc.Driver
jdbcUrl: jdbc:mysql://localhost:3307/user?serverTimezone=UTC&characterEncoding=utf8&characterSetResults=utf8&autoReconnect=true&failOverReadOnly=false
username: root
password: pengwz
props:
sql.show: true
query.with.cipher.column: true
executor.size: 8
max.connections.size.per.query: 1
check.table.metadata.enabled: false
sharding:
defaultDataSourceName: ds0
default-database-strategy:
none:
defaultTableStrategy:
none:
tables:
t_user:
actual-data-nodes: mb_ds0.t_user
table-strategy:
inline:
sharding-column: user_id
algorithm-expression: mb_ds0.t_user
t_order:
database-strategy:
inline:
sharding-column: user_id
algorithm-expression: ma_ds$->{user_id % 2}
actual-data-nodes: ma_ds$->{0..1}.t_order_$->{1..2}
table-strategy:
inline:
sharding-column: order_id
algorithm-expression: t_order_$->{order_id % 2 + 1}
key-generator:
column: id
type: SNOWFLAKE
t_order_item:
actual-data-nodes: ma_ds$->{0..1}.t_order_item_$->{1..2}
databaseStrategy:
inline:
shardingColumn: user_id
algorithmExpression: ma_ds$->{user_id % 2}
table-strategy:
inline:
sharding-column: order_id
algorithm-expression: t_order_item_$->{order_id % 2 + 1}
key-generator:
column: id
type: SNOWFLAKE
binding-tables:
- t_order,t_order_item
broadcast-tables:
- dictionary
master-slave-rules:
ma_ds0:
master-data-source-name: ds0
slave-data-source-names:
- s0
- s1
loadBalanceAlgorithmType: ROUND_ROBIN
ma_ds1:
master-data-source-name: ds1
slave-data-source-names:
- s0
- s1
loadBalanceAlgorithmType: ROUND_ROBIN
mb_ds0:
master-data-source-name: ds2
slave-data-source-names:
- s2
loadBalanceAlgorithmType: ROUND_ROBIN
# When the following encryption configuration is commented out, the sub-database and sub-table strategy will take effect
encrypt:
encryptors:
aes_encryptor:
type: aes
props.aes.key.value: 123456abc
tables:
t_user:
columns:
password:
encryptor: aes_encryptor
plain-column: password
cipher-column: pwd_cipher
encryptor-name: aes_encryptor
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] sandynz commented on issue #6206: SpringBoot-YML configuration, data desensitization and database and table division strategies are not compatible
Posted by GitBox <gi...@apache.org>.
sandynz commented on issue #6206:
URL: https://github.com/apache/shardingsphere/issues/6206#issuecomment-943158170
Hi @pengweizhong , since I haven't found similar issue, how is it going on, have you found the problem cause?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] terrymanu closed issue #6206: SpringBoot-YML configuration, data desensitization and database and table division strategies are not compatible
Posted by GitBox <gi...@apache.org>.
terrymanu closed issue #6206:
URL: https://github.com/apache/shardingsphere/issues/6206
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] pengweizhong commented on issue #6206: SpringBoot-YML configuration, data desensitization and database and table division strategies are not compatible
Posted by GitBox <gi...@apache.org>.
pengweizhong commented on issue #6206:
URL: https://github.com/apache/shardingsphere/issues/6206#issuecomment-651017162
@tristaZero Yes, I have turned it over many times. A single function can be used, such as data desensitization or sub-database sub-table. But when they are used together, it is not allowed.
For example, the current situation: through the console log, you can see that the data desensitization function is normal, but you cannot find the specific physical table when inserting the data. Because the previously used sub-library sub-table strategy can not be used.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] pengweizhong commented on issue #6206: SpringBoot-YML configuration, data desensitization and database and table division strategies are not compatible
Posted by GitBox <gi...@apache.org>.
pengweizhong commented on issue #6206:
URL: https://github.com/apache/shardingsphere/issues/6206#issuecomment-651551257
Regarding the configuration level, I have tried many times in different situations. According to the configuration you said, SPRINGBOOT-YML does not recognize it directly, and the data desensitization does not take effect after the program is run.
Existing configuration:
```yml
spring:
shardingsphere:
encrypt:
. . .
```
Data desensitization takes effect, sub-database sub-table does not take effect
```yml
spring:
shardingsphere:
sharding:
encrypt:
. . .
```
Data desensitization does not take effect, the sub-database sub-table takes effect. In addition, IDEA will prompt that _"this part of the configuration does not exist"_. What is the problem? Or is Sharding not friendly enough for SPRINGBOOT-YML? Should I choose YML configuration or PROPERTIES configuration in a production environment?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] pengweizhong commented on issue #6206: SpringBoot-YML configuration, data desensitization and database and table division strategies are not compatible
Posted by GitBox <gi...@apache.org>.
pengweizhong commented on issue #6206:
URL: https://github.com/apache/shardingsphere/issues/6206#issuecomment-651031923
@tristaZero
Hello, the following is the output of the console when data desensitization is configured and not configured.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] terrymanu commented on issue #6206: SpringBoot-YML configuration, data desensitization and database and table division strategies are not compatible
Posted by GitBox <gi...@apache.org>.
terrymanu commented on issue #6206:
URL: https://github.com/apache/shardingsphere/issues/6206#issuecomment-997433649
I just close the issue because of no response anymore.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] tristaZero commented on issue #6206: SpringBoot-YML configuration, data desensitization and database and table division strategies are not compatible
Posted by GitBox <gi...@apache.org>.
tristaZero commented on issue #6206:
URL: https://github.com/apache/shardingsphere/issues/6206#issuecomment-651012688
@pengweizhong Hi have you ever looked at [sharding+encryption configuration example](https://shardingsphere.apache.org/document/legacy/4.x/document/cn/manual/sharding-jdbc/configuration/config-spring-boot/#data-sharding--data-masking)?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] tristaZero commented on issue #6206: SpringBoot-YML configuration, data desensitization and database and table division strategies are not compatible
Posted by GitBox <gi...@apache.org>.
tristaZero commented on issue #6206:
URL: https://github.com/apache/shardingsphere/issues/6206#issuecomment-651050186
`ShardingRule` is supposed to include `EncryptRule`. However, from your configuration, it looks like `ShardingRule` and `EncryptRule` are at the same level. Please double check.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org