You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2022/07/24 17:39:34 UTC
[GitHub] [shardingsphere] isHuangXin opened a new pull request, #19509: CREATE SHARDING TABLE RULE
isHuangXin opened a new pull request, #19509:
URL: https://github.com/apache/shardingsphere/pull/19509
CREATE SHARDING TABLE RULE #17939.
In this PR, I finished parsing the CREATE SHARDING TABLE RULE in the yaml file and generating the corresponding distsql statement.
The solution for this PR is to first create the sharding algorithm, and then specify the sharding algorithm to create standard sharding rules
**Finally, the result of running my code is**
- The converted distsql output
- Running the converted distsql in SS proxy
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] RaigorJiang commented on pull request #19509: CREATE SHARDING TABLE RULE
Posted by GitBox <gi...@apache.org>.
RaigorJiang commented on PR #19509:
URL: https://github.com/apache/shardingsphere/pull/19509#issuecomment-1195745017
@isHuangXin Please add test cases later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] codecov-commenter commented on pull request #19509: CREATE SHARDING TABLE RULE
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #19509:
URL: https://github.com/apache/shardingsphere/pull/19509#issuecomment-1193561983
# [Codecov](https://codecov.io/gh/apache/shardingsphere/pull/19509?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#19509](https://codecov.io/gh/apache/shardingsphere/pull/19509?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (0dc7ee4) into [master](https://codecov.io/gh/apache/shardingsphere/commit/d1635ce48ceef6a0b9559a49472de1506ebc0796?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d1635ce) will **decrease** coverage by `0.07%`.
> The diff coverage is `0.00%`.
```diff
@@ Coverage Diff @@
## master #19509 +/- ##
============================================
- Coverage 60.11% 60.04% -0.08%
Complexity 2393 2393
============================================
Files 3830 3830
Lines 54478 54554 +76
Branches 7615 7626 +11
============================================
+ Hits 32752 32755 +3
- Misses 18918 18991 +73
Partials 2808 2808
```
| [Impacted Files](https://codecov.io/gh/apache/shardingsphere/pull/19509?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ql/ral/common/constant/DistSQLScriptConstants.java](https://codecov.io/gh/apache/shardingsphere/pull/19509/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtcHJveHkvc2hhcmRpbmdzcGhlcmUtcHJveHktYmFja2VuZC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc2hhcmRpbmdzcGhlcmUvcHJveHkvYmFja2VuZC90ZXh0L2Rpc3RzcWwvcmFsL2NvbW1vbi9jb25zdGFudC9EaXN0U1FMU2NyaXB0Q29uc3RhbnRzLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...ral/queryable/ConvertYamlConfigurationHandler.java](https://codecov.io/gh/apache/shardingsphere/pull/19509/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtcHJveHkvc2hhcmRpbmdzcGhlcmUtcHJveHktYmFja2VuZC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc2hhcmRpbmdzcGhlcmUvcHJveHkvYmFja2VuZC90ZXh0L2Rpc3RzcWwvcmFsL3F1ZXJ5YWJsZS9Db252ZXJ0WWFtbENvbmZpZ3VyYXRpb25IYW5kbGVyLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...nd/text/distsql/ral/hint/enums/HintSourceType.java](https://codecov.io/gh/apache/shardingsphere/pull/19509/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtcHJveHkvc2hhcmRpbmdzcGhlcmUtcHJveHktYmFja2VuZC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc2hhcmRpbmdzcGhlcmUvcHJveHkvYmFja2VuZC90ZXh0L2Rpc3RzcWwvcmFsL2hpbnQvZW51bXMvSGludFNvdXJjZVR5cGUuamF2YQ==) | `42.85% <0.00%> (+42.85%)` | :arrow_up: |
Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] RaigorJiang merged pull request #19509: CREATE SHARDING TABLE RULE
Posted by GitBox <gi...@apache.org>.
RaigorJiang merged PR #19509:
URL: https://github.com/apache/shardingsphere/pull/19509
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] RaigorJiang commented on a diff in pull request #19509: CREATE SHARDING TABLE RULE
Posted by GitBox <gi...@apache.org>.
RaigorJiang commented on code in PR #19509:
URL: https://github.com/apache/shardingsphere/pull/19509#discussion_r928384630
##########
shardingsphere-proxy/shardingsphere-proxy-backend/src/main/java/org/apache/shardingsphere/proxy/backend/text/distsql/ral/common/constant/DistSQLScriptConstants.java:
##########
@@ -63,4 +63,31 @@ public final class DistSQLScriptConstants {
+ ")";
public static final String PROPERTY = "\"%s\"=\"%s\"";
+
+ public static final String CREATE_SHARDING_ALGORITHM = "CREATE SHARDING ALGORITHM";
+
+ public static final String SHARDING_ALGORITHM = " %s ("
+ + System.lineSeparator()
+ + "TYPE(NAME=%s, PROPERTIES(%s))"
+ + System.lineSeparator()
+ + ")";
+
+ public static final String CREATE_SHARDING_TABLE = "CREATE SHARDING TABLE RULE";
+
+ public static final String SHARDING_TABLE = " %s ("
+ + System.lineSeparator()
+ + "DATANODES(\"%s\"),"
+ + System.lineSeparator()
+ + "%s"
+ + System.lineSeparator()
+ + ")";
+
+ public static final String DATABASE_SHARDING_STRATEGY = "DATABASE_STRATEGY(TYPE=standard, SHARDING_COLUMN=%s, SHARDING_ALGORITHM=%s),"
Review Comment:
Completing this feature requires you to be familiar with YAML configuration:
https://shardingsphere.apache.org/document/current/en/user-manual/shardingsphere-jdbc/yaml-config/rules/sharding/
Note that when TYPE is `COMPLEX`, the key is no longer `shardingColumn`, but `shardingColumns`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] RaigorJiang commented on a diff in pull request #19509: CREATE SHARDING TABLE RULE
Posted by GitBox <gi...@apache.org>.
RaigorJiang commented on code in PR #19509:
URL: https://github.com/apache/shardingsphere/pull/19509#discussion_r928383764
##########
shardingsphere-proxy/shardingsphere-proxy-backend/src/main/java/org/apache/shardingsphere/proxy/backend/text/distsql/ral/common/constant/DistSQLScriptConstants.java:
##########
@@ -63,4 +63,31 @@ public final class DistSQLScriptConstants {
+ ")";
public static final String PROPERTY = "\"%s\"=\"%s\"";
+
+ public static final String CREATE_SHARDING_ALGORITHM = "CREATE SHARDING ALGORITHM";
+
+ public static final String SHARDING_ALGORITHM = " %s ("
+ + System.lineSeparator()
+ + "TYPE(NAME=%s, PROPERTIES(%s))"
+ + System.lineSeparator()
+ + ")";
+
+ public static final String CREATE_SHARDING_TABLE = "CREATE SHARDING TABLE RULE";
+
+ public static final String SHARDING_TABLE = " %s ("
+ + System.lineSeparator()
+ + "DATANODES(\"%s\"),"
+ + System.lineSeparator()
+ + "%s"
+ + System.lineSeparator()
+ + ")";
+
+ public static final String DATABASE_SHARDING_STRATEGY = "DATABASE_STRATEGY(TYPE=standard, SHARDING_COLUMN=%s, SHARDING_ALGORITHM=%s),"
Review Comment:
TYPE is not fixed as `standard`, please get it from configuration.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] RaigorJiang commented on a diff in pull request #19509: CREATE SHARDING TABLE RULE
Posted by GitBox <gi...@apache.org>.
RaigorJiang commented on code in PR #19509:
URL: https://github.com/apache/shardingsphere/pull/19509#discussion_r928386524
##########
shardingsphere-proxy/shardingsphere-proxy-backend/src/main/java/org/apache/shardingsphere/proxy/backend/text/distsql/ral/queryable/ConvertYamlConfigurationHandler.java:
##########
@@ -135,4 +144,94 @@ private void appendProperties(final Map<String, Object> properties, final String
}
}
}
+
+ private void appendRules(final Collection<YamlRuleConfiguration> rules, final StringBuilder stringBuilder) {
+ if (rules.isEmpty()) {
+ return;
+ }
+ for (YamlRuleConfiguration rule: rules) {
+ ShardingRuleConfiguration shardingRuleConfig = new ShardingRuleConfigurationYamlSwapper().swapToObject((YamlShardingRuleConfiguration) rule);
+ appendShardingAlgorithm(shardingRuleConfig, stringBuilder);
+ appendShardingTableRules(shardingRuleConfig, stringBuilder);
+ appendShardingBindingTableRules(shardingRuleConfig, stringBuilder);
+ }
+ }
+
+ private void appendShardingAlgorithm(final ShardingRuleConfiguration shardingRuleConfig, final StringBuilder stringBuilder) {
+ stringBuilder.append(DistSQLScriptConstants.CREATE_SHARDING_ALGORITHM);
+ Iterator<Entry<String, ShardingSphereAlgorithmConfiguration>> iterator = shardingRuleConfig.getShardingAlgorithms().entrySet().iterator();
+ while (iterator.hasNext()) {
+ Entry<String, ShardingSphereAlgorithmConfiguration> entry = iterator.next();
+ String shardingAlgorithmName = entry.getKey();
+ String algorithmType = entry.getValue().getType().toLowerCase();
+ String property = appendShardingAlgorithmProperties(entry.getValue().getProps());
+ stringBuilder.append(String.format(DistSQLScriptConstants.SHARDING_ALGORITHM, shardingAlgorithmName, algorithmType, property));
+ if (iterator.hasNext()) {
+ stringBuilder.append(DistSQLScriptConstants.COMMA);
+ }
+ }
+ stringBuilder.append(DistSQLScriptConstants.SEMI).append(System.lineSeparator());
+ }
+
+ private String appendShardingAlgorithmProperties(final Properties property) {
+ StringBuilder result = new StringBuilder();
+ Iterator<Entry<Object, Object>> iterator = property.entrySet().iterator();
+ while (iterator.hasNext()) {
+ Entry<Object, Object> entry = iterator.next();
+ result.append(String.format(DistSQLScriptConstants.PROPERTY, entry.getKey(), entry.getValue()));
+ if (iterator.hasNext()) {
+ result.append(DistSQLScriptConstants.COMMA);
+ }
+ }
+ return result.toString();
+ }
+
+ private void appendShardingTableRules(final ShardingRuleConfiguration shardingRuleConfig, final StringBuilder stringBuilder) {
+ stringBuilder.append(DistSQLScriptConstants.CREATE_SHARDING_TABLE);
+ Iterator<ShardingTableRuleConfiguration> iterator = shardingRuleConfig.getTables().iterator();
Review Comment:
Also need to consider autoTables. shardingRuleConfig.getAutoTables()
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] RaigorJiang commented on pull request #19509: CREATE SHARDING TABLE RULE
Posted by GitBox <gi...@apache.org>.
RaigorJiang commented on PR #19509:
URL: https://github.com/apache/shardingsphere/pull/19509#issuecomment-1196186999
> Get it, but I plan to add test after implementing all functions.
Your PR has decreased coverage a lot, and if continues like this, future PRs may not be merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] RaigorJiang commented on a diff in pull request #19509: CREATE SHARDING TABLE RULE
Posted by GitBox <gi...@apache.org>.
RaigorJiang commented on code in PR #19509:
URL: https://github.com/apache/shardingsphere/pull/19509#discussion_r928385480
##########
shardingsphere-proxy/shardingsphere-proxy-backend/src/main/java/org/apache/shardingsphere/proxy/backend/text/distsql/ral/queryable/ConvertYamlConfigurationHandler.java:
##########
@@ -135,4 +144,94 @@ private void appendProperties(final Map<String, Object> properties, final String
}
}
}
+
+ private void appendRules(final Collection<YamlRuleConfiguration> rules, final StringBuilder stringBuilder) {
+ if (rules.isEmpty()) {
+ return;
+ }
+ for (YamlRuleConfiguration rule: rules) {
+ ShardingRuleConfiguration shardingRuleConfig = new ShardingRuleConfigurationYamlSwapper().swapToObject((YamlShardingRuleConfiguration) rule);
+ appendShardingAlgorithm(shardingRuleConfig, stringBuilder);
+ appendShardingTableRules(shardingRuleConfig, stringBuilder);
+ appendShardingBindingTableRules(shardingRuleConfig, stringBuilder);
+ }
+ }
+
+ private void appendShardingAlgorithm(final ShardingRuleConfiguration shardingRuleConfig, final StringBuilder stringBuilder) {
+ stringBuilder.append(DistSQLScriptConstants.CREATE_SHARDING_ALGORITHM);
+ Iterator<Entry<String, ShardingSphereAlgorithmConfiguration>> iterator = shardingRuleConfig.getShardingAlgorithms().entrySet().iterator();
+ while (iterator.hasNext()) {
+ Entry<String, ShardingSphereAlgorithmConfiguration> entry = iterator.next();
+ String shardingAlgorithmName = entry.getKey();
+ String algorithmType = entry.getValue().getType().toLowerCase();
+ String property = appendShardingAlgorithmProperties(entry.getValue().getProps());
+ stringBuilder.append(String.format(DistSQLScriptConstants.SHARDING_ALGORITHM, shardingAlgorithmName, algorithmType, property));
+ if (iterator.hasNext()) {
+ stringBuilder.append(DistSQLScriptConstants.COMMA);
+ }
+ }
+ stringBuilder.append(DistSQLScriptConstants.SEMI).append(System.lineSeparator());
+ }
+
+ private String appendShardingAlgorithmProperties(final Properties property) {
+ StringBuilder result = new StringBuilder();
+ Iterator<Entry<Object, Object>> iterator = property.entrySet().iterator();
+ while (iterator.hasNext()) {
+ Entry<Object, Object> entry = iterator.next();
+ result.append(String.format(DistSQLScriptConstants.PROPERTY, entry.getKey(), entry.getValue()));
+ if (iterator.hasNext()) {
+ result.append(DistSQLScriptConstants.COMMA);
+ }
+ }
+ return result.toString();
+ }
+
+ private void appendShardingTableRules(final ShardingRuleConfiguration shardingRuleConfig, final StringBuilder stringBuilder) {
+ stringBuilder.append(DistSQLScriptConstants.CREATE_SHARDING_TABLE);
+ Iterator<ShardingTableRuleConfiguration> iterator = shardingRuleConfig.getTables().iterator();
+ while (iterator.hasNext()) {
+ ShardingTableRuleConfiguration entry = iterator.next();
+ String tableName = entry.getLogicTable();
+ String dataNodes = entry.getActualDataNodes();
Review Comment:
If autoTable, here is `RESOURCES`, not `DATANODES`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] isHuangXin commented on a diff in pull request #19509: CREATE SHARDING TABLE RULE
Posted by GitBox <gi...@apache.org>.
isHuangXin commented on code in PR #19509:
URL: https://github.com/apache/shardingsphere/pull/19509#discussion_r928432444
##########
shardingsphere-proxy/shardingsphere-proxy-backend/src/main/java/org/apache/shardingsphere/proxy/backend/text/distsql/ral/common/constant/DistSQLScriptConstants.java:
##########
@@ -63,4 +63,31 @@ public final class DistSQLScriptConstants {
+ ")";
public static final String PROPERTY = "\"%s\"=\"%s\"";
+
+ public static final String CREATE_SHARDING_ALGORITHM = "CREATE SHARDING ALGORITHM";
+
+ public static final String SHARDING_ALGORITHM = " %s ("
+ + System.lineSeparator()
+ + "TYPE(NAME=%s, PROPERTIES(%s))"
+ + System.lineSeparator()
+ + ")";
+
+ public static final String CREATE_SHARDING_TABLE = "CREATE SHARDING TABLE RULE";
+
+ public static final String SHARDING_TABLE = " %s ("
+ + System.lineSeparator()
+ + "DATANODES(\"%s\"),"
+ + System.lineSeparator()
+ + "%s"
+ + System.lineSeparator()
+ + ")";
+
+ public static final String DATABASE_SHARDING_STRATEGY = "DATABASE_STRATEGY(TYPE=standard, SHARDING_COLUMN=%s, SHARDING_ALGORITHM=%s),"
Review Comment:
> TYPE is not fixed as `standard`, please get it from configuration.
Fixed !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] isHuangXin commented on pull request #19509: CREATE SHARDING TABLE RULE
Posted by GitBox <gi...@apache.org>.
isHuangXin commented on PR #19509:
URL: https://github.com/apache/shardingsphere/pull/19509#issuecomment-1195746787
> @isHuangXin Please add test cases later.
Get it, but I plan to add test after implementing all functions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] isHuangXin commented on pull request #19509: CREATE SHARDING TABLE RULE
Posted by GitBox <gi...@apache.org>.
isHuangXin commented on PR #19509:
URL: https://github.com/apache/shardingsphere/pull/19509#issuecomment-1196234224
> > Get it, but I plan to add test after implementing all functions.
>
> Your PR has decreased coverage a lot, and if continues like this, future PRs may not be merged.
OK, I will finish the test unit in the next PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org