You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2022/10/10 06:40:42 UTC
[GitHub] [shardingsphere] natehuangting opened a new issue, #18430: Sharding Audit
natehuangting opened a new issue, #18430:
URL: https://github.com/apache/shardingsphere/issues/18430
## Feature Request
### Describe the feature you would like.
I want to add a configuration in server.yaml called
allow-broadcast-without-sharding-key, it indicates whether to allow broadcasting without sharding key, the default is true, which is consistent with the existing logic, and can be changed to false if necessary.
The broadcast behavior without the shard key will cause a large area of access to the DB, causing timeouts and other problems. If it is an update, the risk is even greater, so I think it is necessary to support it.
I checked the source code, and it seems that as long as I add this logical judgment to the ShardingStandardRoutingEngine, I have already implemented it.
I would like to consult my requirements, parameter naming and whether the location is reasonable.
Hope for your reply!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] natehuangting commented on issue #18430: Queries and updates without shard keys are not allowed
Posted by GitBox <gi...@apache.org>.
natehuangting commented on issue #18430:
URL: https://github.com/apache/shardingsphere/issues/18430#issuecomment-1168122045
I have designed the audit scheme as follows.
The first is configuration, like ShardingShpereAlgorithm, only type and props, so it can also be defined using ShardingShpereAlgorithm.
I thought of two configuration methods, scheme 1 is more brief, scheme 2 is more readable.It should be noted that the algorithm will only be defined and used only once in the Sharding configuration.
scheme 1
```
rules:
- !SHARDING
audits:
- type: SHARDING_KEY_CHECK
props:
allow-hint-without_sharding_key: true
- type: SHARDING_KEY_CHECK
props:
allow-hint-without_sharding_key: true
```
Scheme 2
```
rules:
- !SHARDING
auditAlgorithms:
default_audit1:
type: SHARDING_KEY_CHECK
props:
allow-hint-without_sharding_key: true
default_audit2:
type: SHARDING_KEY_CHECK
props:
allow-hint-without_sharding_key: true
audits:
- default_audit1
- default_audit2
```
```
package org.apache.shardingsphere.sharding.yaml.config;
public final class YamlShardingRuleConfiguration implements YamlRuleConfiguration {
private Collection<YamlShardingSphereAlgorithmConfiguration> audits = new LinkedList<>();
}
```
The second is ShardingAuditAlgorithm, like ShardingAlgorithm, ShardingAuditAlgorithm is also a subclass of ShardingSphereAlgorithm.
<img width="297" alt="image" src="https://user-images.githubusercontent.com/26433395/176070140-87075b21-dce1-4e59-953e-4a5abc5d98f1.png">
```
package org.apache.shardingsphere.sharding.spi;
public final class ShardingAuditChecker implements SQLChecker<ShardingRule> {
@Override
public boolean check(final String databaseName, final Grantee grantee, final ShardingRule rule) {
return true;
}
@Override
public SQLCheckResult check(final SQLStatement sqlStatement, final List<Object> parameters, final Grantee grantee,
final String currentDatabase, final Map<String, ShardingSphereDatabase> databases,
final ShardingRule rule) {
for (ShardingAuditAlgorithm shardingAuditCheckAlgorithm : rule.getShardingAuditAlgorithms()) {
SQLCheckResult sqlCheckResult = shardingAuditCheckAlgorithm.check(sqlStatement, parameters, grantee, currentDatabase, databases, rule);
if (!sqlCheckResult.isPassed()) {
return sqlCheckResult;
}
}
return new SQLCheckResult(true, "");
}
@Override
public boolean check(final Grantee grantee, final ShardingRule rule) {
return true;
}
@Override
public boolean check(final Grantee grantee, final BiPredicate<Object, Object> validator, final Object cipher,
final ShardingRule rule) {
return true;
}
@Override
public int getOrder() {
return ShardingAuditOrder.ORDER;
}
@Override
public Class<ShardingRule> getTypeClass() {
return ShardingRule.class;
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] strongduanmu commented on issue #18430: Queries and updates without shard keys are not allowed
Posted by GitBox <gi...@apache.org>.
strongduanmu commented on issue #18430:
URL: https://github.com/apache/shardingsphere/issues/18430#issuecomment-1159878478
Hi @natehuangting, thank you for your feedback, I think this is a really great suggestion. Besides, ShardingStatementValidator might be an ideal place to implement this feature.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] dongzl commented on issue #18430: Sharding Audit
Posted by GitBox <gi...@apache.org>.
dongzl commented on issue #18430:
URL: https://github.com/apache/shardingsphere/issues/18430#issuecomment-1222251908
> This issue has four remaining subtasks:
>
> * [ ] support for spring namespace and spring boot starter
> * [ ] add new DistSQL for audit
> * [ ] document
> * [ ] test case
>
> Is anyone interested in doing these tasks?
I will try it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] strongduanmu commented on issue #18430: Sharding Audit
Posted by GitBox <gi...@apache.org>.
strongduanmu commented on issue #18430:
URL: https://github.com/apache/shardingsphere/issues/18430#issuecomment-1341853481
All tasks has been finished.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] terrymanu commented on issue #18430: Sharding Audit
Posted by GitBox <gi...@apache.org>.
terrymanu commented on issue #18430:
URL: https://github.com/apache/shardingsphere/issues/18430#issuecomment-1174567620
How about change auditAlgorithms to auditors?
It is better to keep consist with keyGenerators' name style.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] terrymanu closed issue #18430: Queries and updates without shard keys are not allowed
Posted by GitBox <gi...@apache.org>.
terrymanu closed issue #18430: Queries and updates without shard keys are not allowed
URL: https://github.com/apache/shardingsphere/issues/18430
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] github-actions[bot] closed issue #18430: Sharding Audit
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed issue #18430: Sharding Audit
URL: https://github.com/apache/shardingsphere/issues/18430
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] strongduanmu commented on issue #18430: Sharding Audit
Posted by GitBox <gi...@apache.org>.
strongduanmu commented on issue #18430:
URL: https://github.com/apache/shardingsphere/issues/18430#issuecomment-1222217209
This issue also has four subtasks:
- [ ] support for spring namespace and spring boot starter
- [ ] add new DistSQL for audit
- [ ] document
- [ ] test case
Is anyone interested in doing these tasks?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] natehuangting commented on issue #18430: Sharding Audit
Posted by GitBox <gi...@apache.org>.
natehuangting commented on issue #18430:
URL: https://github.com/apache/shardingsphere/issues/18430#issuecomment-1174703804
I will refactor config auditAlgorithms to auditors，auditAlgorithmNames to auditorNames.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] dongzl commented on issue #18430: Sharding Audit
Posted by GitBox <gi...@apache.org>.
dongzl commented on issue #18430:
URL: https://github.com/apache/shardingsphere/issues/18430#issuecomment-1245140485
create sharding table rule with audit strategy:
```
CREATE SHARDING TABLE RULE t_order (
DATANODES("ds_${0..1}.t_order_${0..2}"),
TABLE_STRATEGY(TYPE=STANDARD,SHARDING_COLUMN=order_id,SHARDING_ALGORITHM=t_order_inline),
KEY_GENERATE_STRATEGY(COLUMN=order_id,KEY_GENERATOR=snowflake_key_generator),
AUDIT_STRATEGY(AUDITORS=[auditor1,auditor2],ALLOW_HINT_DISABLE=true)
);
```
create sharding table rule with audit definition:
````
CREATE SHARDING TABLE RULE t_order_detail (
DATANODES("ds_${0..1}.t_order_detail_${0..1}"),
DATABASE_STRATEGY(TYPE=STANDARD,SHARDING_COLUMN=user_id,SHARDING_ALGORITHM(TYPE(NAME=INLINE,PROPERTIES("algorithm-expression"="ds_${user_id % 2}")))),
TABLE_STRATEGY(TYPE=STANDARD,SHARDING_COLUMN=order_id,SHARDING_ALGORITHM(TYPE(NAME=INLINE,PROPERTIES("algorithm-expression"="t_order_detail_${order_id % 3}")))),
KEY_GENERATE_STRATEGY(COLUMN=detail_id,TYPE(NAME=snowflake)),
AUDIT_STRATEGY([(NAME=auditor1,TYPE(NAME=DML_SHARDING_CONDITIONS)),(NAME=auditor2,TYPE(NAME=DML_SHARDING_CONDITIONS))],ALLOW_HINT_DISABLE=true)
);
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] strongduanmu commented on issue #18430: Sharding Audit
Posted by GitBox <gi...@apache.org>.
strongduanmu commented on issue #18430:
URL: https://github.com/apache/shardingsphere/issues/18430#issuecomment-1222252571
@dongzl Welcome.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] github-actions[bot] commented on issue #18430: Sharding Audit
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #18430:
URL: https://github.com/apache/shardingsphere/issues/18430#issuecomment-1272349596
Hello , this issue has not received a reply for several days.
This issue is supposed to be closed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] strongduanmu closed issue #18430: Sharding Audit
Posted by GitBox <gi...@apache.org>.
strongduanmu closed issue #18430: Sharding Audit
URL: https://github.com/apache/shardingsphere/issues/18430
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org