You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2020/09/16 02:47:12 UTC
[GitHub] [shardingsphere] johnny2002 opened a new issue #7471: Simplify configuration file
johnny2002 opened a new issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471
Currently, shardingsphere configuration file need to repeat all tables' config, event they are using same sharding rule.
e.g.
```properties
spring.shardingsphere.sharding.tables.table1.actual-data-nodes=ds$->{0..1}.tab_user$->{0..1}
spring.shardingsphere.sharding.tables.table1.table-strategy.inline.sharding-column=id
spring.shardingsphere.sharding.tables.table1.table-strategy.inline.algorithm-expression=tab_user$->{id % 2}
spring.shardingsphere.sharding.tables.table2.actual-data-nodes=ds$->{0..1}.tab_user$->{0..1}
spring.shardingsphere.sharding.tables.table2.table-strategy.inline.sharding-column=id
spring.shardingsphere.sharding.tables.table2.table-strategy.inline.algorithm-expression=tab_user$->{id % 2}
```
in the example, table1 and table2 have exactly same sharding rule, I suggest to combine them into one group, like:
```properties
spring.shardingsphere.sharding.groups.group1.tables=table1,table2
spring.shardingsphere.sharding.groups.group1.actual-data-nodes=ds$->{0..1}.tab_user$->{0..1}
spring.shardingsphere.sharding.tables.group1.table-strategy.inline.sharding-column=id
spring.shardingsphere.sharding.tables.group1.table-strategy.inline.algorithm-expression=tab_user$->{id % 2}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 closed issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 closed issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 commented on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 commented on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-694691019
Hi Trista,
with shardingAlgorithms, we still need need to repeat lines like following:
t_order:
actualDataNodes: ds_${0..1}.t_order_${0..1}
tableStrategy:
standard:
shardingColumn: order_id
shardingAlgorithmName: t_order_inline
I feel it's still a little complex.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 commented on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 commented on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-699789167
> ```yaml
> rules:
> - !SHARDING
> tables:
> template_item_id:
> t_order:
> ```
Hi Trista,
Got it, I will working on datasource now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 commented on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 commented on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-696648036
> Hi @johnny2002
> Your suggestion about `dataSourceTemplates` got some positive feedback from others.
>
> Before any programming, another important prerequisite is to design the configuration of [Spring NameSpace](https://github.com/apache/shardingsphere/blob/master/shardingsphere-features/shardingsphere-sharding/shardingsphere-sharding-spring/shardingsphere-sharding-spring-namespace/src/main/resources/META-INF/namespace/sharding.xsd).
>
> **Welcome your innovative idea**!
>
> BTW, are you interested in this issue？If so welcome your joining. Otherwise, I will make it involved in the community.
Trista，
I absolutely interested to join your team. Thanks a lot.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 commented on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 commented on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-698884084
> @johnny2002 Sorry for my late reply.
>
> I assigned this issue to you, which means you are in charge of this feature, welcome!
> From my perspective, here are the brief procedure of this issue FYI.
>
> * [x] Design the configuration of `YAML`
> * [ ] Design the configuration API of [Spring NameSpace](https://github.com/apache/shardingsphere/blob/master/shardingsphere-features/shardingsphere-sharding/shardingsphere-sharding-spring/shardingsphere-sharding-spring-namespace/src/main/resources/META-INF/namespace/sharding.xsd) for our next discussion.
> * [ ] Coding work to modify the configuration handling of `YAML`, `Java-based` for `ShardingJDBC` and `ShardingProxy`
> * [ ] Coding work to modify the configuration handling of `SpringNameSpace`, `SpringBoot` for `ShardingJDBC`
> * [ ] The similar changes for `ShardingSphere example module`
> * [ ] The API doc
>
> This is just a crude plan for this issue. However, you are the exact person responsible for it on-site, so follow your pace!
> One concern for me is that you had better make one PR **not so large**.
> If you need any help, please be free to ping me. :-)
Thanks for your trust. I am working on it now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] tristaZero commented on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
tristaZero commented on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-698738541
@johnny2002 Sorry for my late reply.
I assigned this issue to you, which means you are in charge of this feature, welcome!
From my perspective, here are the brief procedure of this issue FYI.
- [x] Design the configuration of `YAML`
- [ ] Design the configuration API of [Spring NameSpace](https://github.com/apache/shardingsphere/blob/master/shardingsphere-features/shardingsphere-sharding/shardingsphere-sharding-spring/shardingsphere-sharding-spring-namespace/src/main/resources/META-INF/namespace/sharding.xsd) for our next discussion.
- [ ] Coding work to modify the configuration handling of `YAML`, `Java-based` for `ShardingJDBC` and `ShardingProxy`
- [ ] Coding work to modify the configuration handling of `SpringNameSpace`, `SpringBoot` for `ShardingJDBC`
- [ ] The similar changes for `ShardingSphere example module`
- [ ] The API doc
This is just a crude plan for this issue. However, you are the exact person responsible for it on-site, so follow your pace!
One concern for me is that you had better make one PR **not so large**.
If you need any help, please be free to ping me. :-)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] tristaZero commented on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
tristaZero commented on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-695787679
In some cases, it is indeed. On the other hand, if you want to use the same sharding algorithm many times, then a reference name is simpler, IMO.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 edited a comment on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 edited a comment on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-699625031
> @johnny2002 Sorry for my late reply.
>
> I assigned this issue to you, which means you are in charge of this feature, welcome!
> From my perspective, here are the brief procedure of this issue FYI.
>
> * [x] Design the configuration of `YAML`
> * [ ] Design the configuration API of [Spring NameSpace](https://github.com/apache/shardingsphere/blob/master/shardingsphere-features/shardingsphere-sharding/shardingsphere-sharding-spring/shardingsphere-sharding-spring-namespace/src/main/resources/META-INF/namespace/sharding.xsd) for our next discussion.
> * [ ] Coding work to modify the configuration handling of `YAML`, `Java-based` for `ShardingJDBC` and `ShardingProxy`
> * [ ] Coding work to modify the configuration handling of `SpringNameSpace`, `SpringBoot` for `ShardingJDBC`
> * [ ] The similar changes for `ShardingSphere example module`
> * [ ] The API doc
>
> This is just a crude plan for this issue. However, you are the exact person responsible for it on-site, so follow your pace!
> One concern for me is that you had better make one PR **not so large**.
> If you need any help, please be free to ping me. :-)
@tristaZero
Hi Trista,
Here is a complete sample of proposed YAML configuration file:
[config-sharding.txt](https://github.com/apache/shardingsphere/files/5288217/config-sharding.txt)
Please review and feedback.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 edited a comment on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 edited a comment on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-693187128
> Hi @johnny2002
> `actual-data-nodes` expresses the positions of actual tables. In general, `logic_table_a` and `logic_table_b` have the same table rule though, the `actual-data-nodes` is different, right?
>
> BTW, `defaultTableRule` seems to fit your case. What do you think? Plus, we gave a huge improvement in the upcoming 5.x release configuration, like the following example.
>
> ```yaml
> schemaName: sharding_db
>
> dataSourceCommon:
> username: root
> password:
> connectionTimeoutMilliseconds: 30000
> idleTimeoutMilliseconds: 60000
> maxLifetimeMilliseconds: 1800000
> maxPoolSize: 50
> minPoolSize: 1
> maintenanceIntervalMilliseconds: 30000
>
> dataSources:
> ds_0:
> url: jdbc:mysql://127.0.0.1:3306/demo_ds_0?serverTimezone=UTC&useSSL=false
> ds_1:
> url: jdbc:mysql://127.0.0.1:3306/demo_ds_1?serverTimezone=UTC&useSSL=false
>
> rules:
> - !SHARDING
> tables:
> t_order:
> actualDataNodes: ds_${0..1}.t_order_${0..1}
> tableStrategy:
> standard:
> shardingColumn: order_id
> shardingAlgorithmName: t_order_inline
> keyGenerateStrategy:
> column: order_id
> keyGeneratorName: snowflake
> t_order_item:
> actualDataNodes: ds_${0..1}.t_order_item_${0..1}
> tableStrategy:
> standard:
> shardingColumn: order_id
> shardingAlgorithmName: t_order_item_inline
> keyGenerateStrategy:
> column: order_item_id
> keyGeneratorName: snowflake
> bindingTables:
> - t_order,t_order_item
> defaultDatabaseStrategy:
> standard:
> shardingColumn: user_id
> shardingAlgorithmName: database_inline
> defaultTableStrategy:
> none:
>
> shardingAlgorithms:
> database_inline:
> type: INLINE
> props:
> algorithm-expression: ds_${user_id % 2}
> t_order_inline:
> type: INLINE
> props:
> algorithm-expression: t_order_${order_id % 2}
> t_order_item_inline:
> type: INLINE
> props:
> algorithm-expression: t_order_item_${order_id % 2}
>
> keyGenerators:
> snowflake:
> type: SNOWFLAKE
> props:
> worker-id: 123
> ```
Hi Trista，
1. actual-data-nodes expresses the positions of actual tables. In general, logic_table_a and logic_table_b have the same table rule though, the actual-data-nodes is different, right?
【Exactly, Yes】
2. defaultTableRule seems to fit your case
【No, defaultTableRule only has one, while I have some different sharding rules】
In my real case, some tables sharding by column "customer_id", some tables sharding by date. even sharding by same column, while they may have different slices. For reference tables and parameter tables, no sharding.
Even some tables sharding with same column, while the sub-tables count may not be same.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 commented on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 commented on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-699725247
'''yaml
rules:
- !SHARDING
tables:
template_item_id: #tables that sharding by item_id
actualDataNodes: ds_${0..1}.${LOGIC_TABLE}_${0..4}
databaseStrategy:
type: INLINE
shardingColumns: item_id
props:
#algorithm-expression: ds_${item_id % 2}
algorithm-expression: ${item_id % 2} #ignore "ds_", since it's already in actualDataNodes
tableStrategy:
type: INLINE
shardingColumns: item_id
props:
algorithm-expression: ${item_id % 5} #ignore "${LOGIC_TABLE}_", since it's already in actualDataNodes
template_order_id: #tables that sharding by order_id
actualDataNodes: ds_${0..1}.${LOGIC_TABLE}${0..9}
#use defaultDatabaseStrategy
tableStrategy:
type: INLINE
shardingColumns: order_id
props:
algorithm-expression: ${order_id % 10} #ignore "${LOGIC_TABLE}_", since it's already in actualDataNodes
t_order:
template: template_order_id
t_order_item:
template: template_order_id
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 commented on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 commented on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-706857937
@tristaZero
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 commented on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 commented on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-706857334
Hi Trista,
PR: https://github.com/apache/shardingsphere/pull/7747
Please review.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 edited a comment on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 edited a comment on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-699725247
This is another way. By this way, we try to compatible with config of old version, and the same style of datasource template.
```yaml
rules:
- !SHARDING
tables:
template_item_id: #tables that sharding by item_id
actualDataNodes: ds_${0..1}.${LOGIC_TABLE}_${0..4}
databaseStrategy:
type: INLINE
shardingColumns: item_id
props:
#algorithm-expression: ds_${item_id % 2}
algorithm-expression: ${item_id % 2} #ignore "ds_", since it's already in actualDataNodes
tableStrategy:
type: INLINE
shardingColumns: item_id
props:
algorithm-expression: ${item_id % 5} #ignore "${LOGIC_TABLE}_", since it's already in actualDataNodes
template_order_id: #tables that sharding by order_id
actualDataNodes: ds_${0..1}.${LOGIC_TABLE}${0..9}
#use defaultDatabaseStrategy
tableStrategy:
type: INLINE
shardingColumns: order_id
props:
algorithm-expression: ${order_id % 10} #ignore "${LOGIC_TABLE}_", since it's already in actualDataNodes
t_order:
template: template_order_id #reference to template_order_id
tableStrategy:
shardingColumns: id # can override the value of tableStrategy.shardingColumns from template.
t_order_item:
template: template_order_id
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 edited a comment on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 edited a comment on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-693191012
For datasource config, how about the following:
```yaml
dataSourceTemplates:
template1:
driver-class-name: com.mysql.jdbc.Driver
username: root
password:
maxPoolSize: 50
minPoolSize: 1
template2:
driver-class-name: org.postgresql.Driver
username: user1
password: pass123
maxPoolSize: 200
minPoolSize: 10
dataSources:
ds_0:
template: template1
url: jdbc:mysql://127.0.0.1:3306/demo_ds_0?serverTimezone=UTC&useSSL=false
ds_1:
template: template1
url: jdbc:mysql://127.0.0.1:3306/demo_ds_1?serverTimezone=UTC&useSSL=false
ds_2:
template: template2
url: jdbc:postgresql://localhost:5432,localhost:5433/postgres?loadBalanceHosts=true&characterEncoding=utf-8
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] tristaZero edited a comment on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
tristaZero edited a comment on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-693144772
Hi @johnny2002
`actual-data-nodes` expresses the positions of actual tables. In general, `logic_table_a` and `logic_table_b` have the same table rule though, the `actual-data-nodes` is different, right?
BTW, `defaultTableRule` seems to fit your case. What do you think? Plus, we gave a huge improvement in the upcoming 5.x release configuration, like the following example.
```yaml
schemaName: sharding_db
dataSourceCommon:
username: root
password:
connectionTimeoutMilliseconds: 30000
idleTimeoutMilliseconds: 60000
maxLifetimeMilliseconds: 1800000
maxPoolSize: 50
minPoolSize: 1
maintenanceIntervalMilliseconds: 30000
dataSources:
ds_0:
url: jdbc:mysql://127.0.0.1:3306/demo_ds_0?serverTimezone=UTC&useSSL=false
ds_1:
url: jdbc:mysql://127.0.0.1:3306/demo_ds_1?serverTimezone=UTC&useSSL=false
rules:
- !SHARDING
tables:
t_order:
actualDataNodes: ds_${0..1}.t_order_${0..1}
tableStrategy:
standard:
shardingColumn: order_id
shardingAlgorithmName: t_order_inline
keyGenerateStrategy:
column: order_id
keyGeneratorName: snowflake
t_order_item:
actualDataNodes: ds_${0..1}.t_order_item_${0..1}
tableStrategy:
standard:
shardingColumn: order_id
shardingAlgorithmName: t_order_item_inline
keyGenerateStrategy:
column: order_item_id
keyGeneratorName: snowflake
bindingTables:
- t_order,t_order_item
defaultDatabaseStrategy:
standard:
shardingColumn: user_id
shardingAlgorithmName: database_inline
defaultTableStrategy:
none:
shardingAlgorithms:
database_inline:
type: INLINE
props:
algorithm-expression: ds_${user_id % 2}
t_order_inline:
type: INLINE
props:
algorithm-expression: t_order_${order_id % 2}
t_order_item_inline:
type: INLINE
props:
algorithm-expression: t_order_item_${order_id % 2}
keyGenerators:
snowflake:
type: SNOWFLAKE
props:
worker-id: 123
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] tristaZero commented on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
tristaZero commented on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-698738541
@johnny2002 Sorry for my late reply.
I assigned this issue to you, which means you are in charge of this feature, welcome!
From my perspective, here are the brief procedure of this issue FYI.
- [x] Design the configuration of `YAML`
- [ ] Design the configuration API of [Spring NameSpace](https://github.com/apache/shardingsphere/blob/master/shardingsphere-features/shardingsphere-sharding/shardingsphere-sharding-spring/shardingsphere-sharding-spring-namespace/src/main/resources/META-INF/namespace/sharding.xsd) for our next discussion.
- [ ] Coding work to modify the configuration handling of `YAML`, `Java-based` for `ShardingJDBC` and `ShardingProxy`
- [ ] Coding work to modify the configuration handling of `SpringNameSpace`, `SpringBoot` for `ShardingJDBC`
- [ ] The similar changes for `ShardingSphere example module`
- [ ] The API doc
This is just a crude plan for this issue. However, you are the exact person responsible for it on-site, so follow your pace!
One concern for me is that you had better make one PR **not so large**.
If you need any help, please be free to ping me. :-)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] tristaZero commented on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
tristaZero commented on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-696553162
Hi @johnny2002
Your suggestion about `dataSourceTemplates` got some positive feedback from others.
Before any programming, another important prerequisite is to design the configuration of [Spring NameSpace](https://github.com/apache/shardingsphere/blob/master/shardingsphere-features/shardingsphere-sharding/shardingsphere-sharding-spring/shardingsphere-sharding-spring-namespace/src/main/resources/META-INF/namespace/sharding.xsd).
**Welcome your innovative idea**!
BTW, are you interested in this issue？If so welcome your joining. Otherwise, I will make it involved in the community.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 edited a comment on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 edited a comment on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-699725247
This is another way. By this way, we try to compatible with config of old version, and the same style of datasource template.
```yaml
rules:
- !SHARDING
tables:
template_item_id: #tables that sharding by item_id
actualDataNodes: ds_${0..1}.${LOGIC_TABLE}_${0..4}
databaseStrategy:
type: INLINE
shardingColumns: item_id
props:
#algorithm-expression: ds_${item_id % 2}
algorithm-expression: ${item_id % 2} #ignore "ds_", since it's already in actualDataNodes
tableStrategy:
type: INLINE
shardingColumns: item_id
props:
algorithm-expression: ${item_id % 5} #ignore "${LOGIC_TABLE}_", since it's already in actualDataNodes
template_order_id: #tables that sharding by order_id
actualDataNodes: ds_${0..1}.${LOGIC_TABLE}${0..9}
#use defaultDatabaseStrategy
tableStrategy:
type: INLINE
shardingColumns: order_id
props:
algorithm-expression: ${order_id % 10} #ignore "${LOGIC_TABLE}_", since it's already in actualDataNodes
t_order:
template: template_order_id #reference to template_order_id
tableStrategy:
shardingColumns: id # can override the value of tableStrategy.shardingColumns from template.
t_order_item:
template: template_order_id
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] tristaZero commented on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
tristaZero commented on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-693144772
Hi @johnny2002
`actual-data-nodes` expresses the positions of actual tables. In general, `logic_table_a` and `logic_table_b` have the same table rule though, the `actual-data-nodes` is different, right?
BTW, `defaultTableRule` seems to fit your case. What do you think? Plus, we gave a huge improvement in the upcoming 5.x release configuration, like the following example.
```yaml
schemaName: sharding_db
dataSourceCommon:
username: postgres
password: postgres
connectionTimeoutMilliseconds: 30000
idleTimeoutMilliseconds: 60000
maxLifetimeMilliseconds: 1800000
maxPoolSize: 50
minPoolSize: 1
maintenanceIntervalMilliseconds: 30000
dataSources:
ds_0:
url: jdbc:postgresql://127.0.0.1:5432/demo_ds_0?serverTimezone=UTC&useSSL=false
ds_1:
url: jdbc:postgresql://127.0.0.1:5432/demo_ds_1?serverTimezone=UTC&useSSL=false
rules:
- !SHARDING
tables:
t_order:
actualDataNodes: ds_${0..1}.t_order_${0..1}
tableStrategy:
standard:
shardingColumn: order_id
shardingAlgorithmName: t_order_inline
keyGenerateStrategy:
column: order_id
keyGeneratorName: snowflake
t_order_item:
actualDataNodes: ds_${0..1}.t_order_item_${0..1}
tableStrategy:
standard:
shardingColumn: order_id
shardingAlgorithmName: t_order_item_inline
keyGenerateStrategy:
column: order_item_id
keyGeneratorName: snowflake
bindingTables:
- t_order,t_order_item
defaultDatabaseStrategy:
standard:
shardingColumn: user_id
shardingAlgorithmName: database_inline
defaultTableStrategy:
none:
shardingAlgorithms:
database_inline:
type: INLINE
props:
algorithm-expression: ds_${user_id % 2}
t_order_inline:
type: INLINE
props:
algorithm-expression: t_order_${order_id % 2}
t_order_item_inline:
type: INLINE
props:
algorithm-expression: t_order_item_${order_id % 2}
keyGenerators:
snowflake:
type: SNOWFLAKE
props:
worker-id: 123
#####################################################################################################
If you want to connect to MySQL, you should manually copy MySQL driver to lib directory.
#####################################################################################################
schemaName: sharding_db
dataSourceCommon:
username: root
password:
connectionTimeoutMilliseconds: 30000
idleTimeoutMilliseconds: 60000
maxLifetimeMilliseconds: 1800000
maxPoolSize: 50
minPoolSize: 1
maintenanceIntervalMilliseconds: 30000
dataSources:
ds_0:
url: jdbc:mysql://127.0.0.1:3306/demo_ds_0?serverTimezone=UTC&useSSL=false
ds_1:
url: jdbc:mysql://127.0.0.1:3306/demo_ds_1?serverTimezone=UTC&useSSL=false
rules:
- !SHARDING
tables:
t_order:
actualDataNodes: ds_${0..1}.t_order_${0..1}
tableStrategy:
standard:
shardingColumn: order_id
shardingAlgorithmName: t_order_inline
keyGenerateStrategy:
column: order_id
keyGeneratorName: snowflake
t_order_item:
actualDataNodes: ds_${0..1}.t_order_item_${0..1}
tableStrategy:
standard:
shardingColumn: order_id
shardingAlgorithmName: t_order_item_inline
keyGenerateStrategy:
column: order_item_id
keyGeneratorName: snowflake
bindingTables:
- t_order,t_order_item
defaultDatabaseStrategy:
standard:
shardingColumn: user_id
shardingAlgorithmName: database_inline
defaultTableStrategy:
none:
shardingAlgorithms:
database_inline:
type: INLINE
props:
algorithm-expression: ds_${user_id % 2}
t_order_inline:
type: INLINE
props:
algorithm-expression: t_order_${order_id % 2}
t_order_item_inline:
type: INLINE
props:
algorithm-expression: t_order_item_${order_id % 2}
keyGenerators:
snowflake:
type: SNOWFLAKE
props:
worker-id: 123
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 edited a comment on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 edited a comment on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-699626194
```yaml
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
dataSourceTemplates:
template1:
driver-class-name: com.mysql.jdbc.Driver
username: root
password:
maxPoolSize: 50
minPoolSize:
template2:
driver-class-name: org.postgresql.Driver
username: user1
password: pass123
maxPoolSize: 200
minPoolSize: 1
dataSources:
ds_0:
template: template1
url: jdbc:mysql://127.0.0.1:3306/demo_ds_0?serverTimezone=UTC&useSSL=false
maxPoolSize: 60 #overrides 50 from template1
ds_1:
template: template1
url: jdbc:mysql://127.0.0.1:3306/demo_ds_1?serverTimezone=UTC&useSSL=false
ds_2:
template: template2
url: jdbc:postgresql://localhost:5432,localhost:5433/postgres?loadBalanceHosts=true&characterEncoding=utf-8
rules:
- !SHARDING
shardingTables:
- #Sample1: all tables Sharding by item_id should be under this group
tables: t_order, t_order_item
actualDataNodes: ds_${0..1}.${LOGIC_TABLE}${0..4} #${LOGIC_TABLE} will be replaced by INLINE strategy implementation
tableStrategy:
type: INLINE #Standard in house implementation
#column: order_id
columns: order_id #use columns to consolidate with "complex" config
#algorithm-expression: t_order${order_id % 5}
algorithm-expression: ${order_id % 5} #logic_table_name already been defined in "actualDataNodes", so no need to repeat it here, just define the expression to calculate ${0..4} in actualDataNodes
- #Sample2: all tables Sharding by month of date should be under this group
tables: t_access_log, t_sec_log
tableStrategy:
actualDataNodes: ds_${0..1}.${LOGIC_TABLE} # tables divided by date, so don't how many tables here.
type: com.ibm.ibank.dmf.shardingshpere.ext.TxDateShardingAlgorithm #Customize sharding implementation
columns: create_date
defaultDatabaseStrategy:
type: INLINE
columns: user_id
#algorithm-expression: ${order_id % 2}
props: #use generic props, we can let developers pass any parameters to the under strategy implementation
algorithm-expression: ${order_id % 2}
defaultTableStrategy:
type: INLINE
columns: order_id
#algorithm-expression: ${order_id % 2}
props:
algorithm-expression: ${order_id % 2}
keyGenerateStrategies:
- #sample1
tables: t_order
column: order_id
#keyGenerator: INCREMENT
keyGenerator: #add type so that keyGenerator can be customized
type: SNOWFLAK
- #sample2
tables: t_order_item
column: order_item_id
keyGenerator:
type: INCREMEN
- #sample3
tables: t_user
column: id
keyGenerator:
type: INCREMEN
autoTables: #not really understand auto-tables currently, will optimize later!!!
-
tables: t_order_auto, t_order_item_auto
actualDataSources: ds_0, ds_1
shardingStrategy:
columns: order_id
shardingAlgorithm:
type: MOD
props:
sharding-count: 2
bindingTables:
- t_order,t_order_item
- t_access_log, t_sec_log
broadcastTables:
- t_config
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 edited a comment on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 edited a comment on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-698884084
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 commented on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 commented on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-699626194
```yaml
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
dataSourceTemplates:
template1:
driver-class-name: com.mysql.jdbc.Driver
username: root
password:
maxPoolSize: 50
minPoolSize:
template2:
driver-class-name: org.postgresql.Driver
username: user1
password: pass123
maxPoolSize: 200
minPoolSize: 1
dataSources:
ds_0:
template: template1
url: jdbc:mysql://127.0.0.1:3306/demo_ds_0?serverTimezone=UTC&useSSL=false
maxPoolSize: 60 #overrides 50 from template1
ds_1:
template: template1
url: jdbc:mysql://127.0.0.1:3306/demo_ds_1?serverTimezone=UTC&useSSL=false
ds_2:
template: template2
url: jdbc:postgresql://localhost:5432,localhost:5433/postgres?loadBalanceHosts=true&characterEncoding=utf-8
rules:
- !SHARDING
shardingTables:
- #Sample1: all tables Sharding by item_id should be under this group
tables: t_order, t_order_item
actualDataNodes: ds_${0..1}.${LOGIC_TABLE}${0..4} #${LOGIC_TABLE} will be replaced by INLINE strategy implementation
tableStrategy:
type: INLINE #Standard in house implementation
#column: order_id
columns: order_id #use columns to consolidate with "complex" config
#algorithm-expression: t_order${order_id % 5}
algorithm-expression: ${order_id % 5} #logic_table_name already been defined in "actualDataNodes", so no need to repeat it here, just define the expression to calculate ${0..4} in actualDataNodes
- #Sample2: all tables Sharding by month of date should be under this group
tables: t_access_log, t_sec_log
tableStrategy:
actualDataNodes: ds_${0..1}.${LOGIC_TABLE} # tables divided by date, so don't how many tables here.
type: com.ibm.ibank.dmf.shardingshpere.ext.TxDateShardingAlgorithm #Customize sharding implementation
columns: create_date
defaultDatabaseStrategy:
type: INLINE
columns: user_id
#algorithm-expression: ${order_id % 2}
props: #use generic props, we can let developers pass any parameters to the under strategy implementation
expression: ${order_id % 2}
defaultTableStrategy:
type: INLINE
columns: order_id
#algorithm-expression: ${order_id % 2}
props:
expression: ${order_id % 2}
keyGenerateStrategies:
- #sample1
tables: t_order
column: order_id
#keyGenerator: INCREMENT
keyGenerator: #add type so that keyGenerator can be customized
type: SNOWFLAK
- #sample2
tables: t_order_item
column: order_item_id
keyGenerator:
type: INCREMEN
- #sample3
tables: t_user
column: id
keyGenerator:
type: INCREMEN
autoTables: #not really understand auto-tables currently, will optimize later!!!
-
tables: t_order_auto, t_order_item_auto
actualDataSources: ds_0, ds_1
shardingStrategy:
columns: order_id
shardingAlgorithm:
type: MOD
props:
sharding-count: 2
bindingTables:
- t_order,t_order_item
- t_access_log, t_sec_log
broadcastTables:
- t_config
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] tristaZero commented on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
tristaZero commented on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-693776052
> I think, you should let developers to create multiple datasource templates. Sometimes, one app may need to connect to difference databases, for an example, business database in oracle and parameter database in mysql.
Oh, that idea seems sort of feasible. I will raize this as a community discussion. Please wait for my news later.
BTW, do you think `shardingAlgorithms:` could solve your issue around some logic tables sharing the same table rule?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 edited a comment on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 edited a comment on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-699725247
This is another way. By this way, we try to compatible with config of old version, and the same style of datasource template.
[config-sharding3.txt](https://github.com/apache/shardingsphere/files/5289664/config-sharding3.txt)
sample:
```yaml
rules:
- !SHARDING
tables:
template_item_id: #tables that sharding by item_id
actualDataNodes: ds_${0..1}.${LOGIC_TABLE}_${0..4}
databaseStrategy:
type: INLINE
shardingColumns: item_id
props:
#algorithm-expression: ds_${item_id % 2}
algorithm-expression: ${item_id % 2} #ignore "ds_", since it's already in actualDataNodes
tableStrategy:
type: INLINE
shardingColumns: item_id
props:
algorithm-expression: ${item_id % 5} #ignore "${LOGIC_TABLE}_", since it's already in actualDataNodes
template_order_id: #tables that sharding by order_id
actualDataNodes: ds_${0..1}.${LOGIC_TABLE}${0..9}
#use defaultDatabaseStrategy
tableStrategy:
type: INLINE
shardingColumns: order_id
props:
algorithm-expression: ${order_id % 10} #ignore "${LOGIC_TABLE}_", since it's already in actualDataNodes
t_order:
template: template_order_id #reference to template_order_id
tableStrategy:
shardingColumns: id # can override the value of tableStrategy.shardingColumns from template.
t_order_item:
template: template_order_id
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 commented on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 commented on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-696648036
> Hi @johnny2002
> Your suggestion about `dataSourceTemplates` got some positive feedback from others.
>
> Before any programming, another important prerequisite is to design the configuration of [Spring NameSpace](https://github.com/apache/shardingsphere/blob/master/shardingsphere-features/shardingsphere-sharding/shardingsphere-sharding-spring/shardingsphere-sharding-spring-namespace/src/main/resources/META-INF/namespace/sharding.xsd).
>
> **Welcome your innovative idea**!
>
> BTW, are you interested in this issue？If so welcome your joining. Otherwise, I will make it involved in the community.
Trista，
I absolutely interested to join your team. Thanks a lot.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 commented on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 commented on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-693191012
For datasource config, how about the following:
```yaml
dataSourceTemplates:
template1:
username: root
password:
maxPoolSize: 50
minPoolSize: 1
template2:
username: user1
password: pass123
maxPoolSize: 200
minPoolSize: 10
dataSources:
ds_0:
template: template1
url: jdbc:mysql://127.0.0.1:3306/demo_ds_0?serverTimezone=UTC&useSSL=false
ds_1:
template: template2
url: jdbc:mysql://127.0.0.1:3306/demo_ds_1?serverTimezone=UTC&useSSL=false
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 edited a comment on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 edited a comment on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-698884084
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 edited a comment on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 edited a comment on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-693187128
> Hi @johnny2002
> `actual-data-nodes` expresses the positions of actual tables. In general, `logic_table_a` and `logic_table_b` have the same table rule though, the `actual-data-nodes` is different, right?
>
> BTW, `defaultTableRule` seems to fit your case. What do you think? Plus, we gave a huge improvement in the upcoming 5.x release configuration, like the following example.
>
> ```yaml
> schemaName: sharding_db
>
> dataSourceCommon:
> username: root
> password:
> connectionTimeoutMilliseconds: 30000
> idleTimeoutMilliseconds: 60000
> maxLifetimeMilliseconds: 1800000
> maxPoolSize: 50
> minPoolSize: 1
> maintenanceIntervalMilliseconds: 30000
>
> dataSources:
> ds_0:
> url: jdbc:mysql://127.0.0.1:3306/demo_ds_0?serverTimezone=UTC&useSSL=false
> ds_1:
> url: jdbc:mysql://127.0.0.1:3306/demo_ds_1?serverTimezone=UTC&useSSL=false
>
> rules:
> - !SHARDING
> tables:
> t_order:
> actualDataNodes: ds_${0..1}.t_order_${0..1}
> tableStrategy:
> standard:
> shardingColumn: order_id
> shardingAlgorithmName: t_order_inline
> keyGenerateStrategy:
> column: order_id
> keyGeneratorName: snowflake
> t_order_item:
> actualDataNodes: ds_${0..1}.t_order_item_${0..1}
> tableStrategy:
> standard:
> shardingColumn: order_id
> shardingAlgorithmName: t_order_item_inline
> keyGenerateStrategy:
> column: order_item_id
> keyGeneratorName: snowflake
> bindingTables:
> - t_order,t_order_item
> defaultDatabaseStrategy:
> standard:
> shardingColumn: user_id
> shardingAlgorithmName: database_inline
> defaultTableStrategy:
> none:
>
> shardingAlgorithms:
> database_inline:
> type: INLINE
> props:
> algorithm-expression: ds_${user_id % 2}
> t_order_inline:
> type: INLINE
> props:
> algorithm-expression: t_order_${order_id % 2}
> t_order_item_inline:
> type: INLINE
> props:
> algorithm-expression: t_order_item_${order_id % 2}
>
> keyGenerators:
> snowflake:
> type: SNOWFLAKE
> props:
> worker-id: 123
> ```
Hi Trista，
1. actual-data-nodes expresses the positions of actual tables. In general, logic_table_a and logic_table_b have the same table rule though, the actual-data-nodes is different, right?
【Exactly, Yes】
2. defaultTableRule seems to fit your case
【No, defaultTableRule only has one, while I have some different sharding rules】
In my real case, some tables sharding by column "customer_id", some tables sharding by date. For some tables, like reference tables and parameter tables, no sharding.
ps. even some tables sharding with same column, while the sub-tables count may not be same.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 edited a comment on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 edited a comment on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-699625031
> @johnny2002 Sorry for my late reply.
>
> I assigned this issue to you, which means you are in charge of this feature, welcome!
> From my perspective, here are the brief procedure of this issue FYI.
>
> * [x] Design the configuration of `YAML`
> * [ ] Design the configuration API of [Spring NameSpace](https://github.com/apache/shardingsphere/blob/master/shardingsphere-features/shardingsphere-sharding/shardingsphere-sharding-spring/shardingsphere-sharding-spring-namespace/src/main/resources/META-INF/namespace/sharding.xsd) for our next discussion.
> * [ ] Coding work to modify the configuration handling of `YAML`, `Java-based` for `ShardingJDBC` and `ShardingProxy`
> * [ ] Coding work to modify the configuration handling of `SpringNameSpace`, `SpringBoot` for `ShardingJDBC`
> * [ ] The similar changes for `ShardingSphere example module`
> * [ ] The API doc
>
> This is just a crude plan for this issue. However, you are the exact person responsible for it on-site, so follow your pace!
> One concern for me is that you had better make one PR **not so large**.
> If you need any help, please be free to ping me. :-)
@tristaZero
Hi Trista,
Here is the complete sample of YAML configuration file:
[config-sharding.txt](https://github.com/apache/shardingsphere/files/5288217/config-sharding.txt)
Please review and feedback.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 edited a comment on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 edited a comment on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-699625031
> @johnny2002 Sorry for my late reply.
>
> I assigned this issue to you, which means you are in charge of this feature, welcome!
> From my perspective, here are the brief procedure of this issue FYI.
>
> * [x] Design the configuration of `YAML`
> * [ ] Design the configuration API of [Spring NameSpace](https://github.com/apache/shardingsphere/blob/master/shardingsphere-features/shardingsphere-sharding/shardingsphere-sharding-spring/shardingsphere-sharding-spring-namespace/src/main/resources/META-INF/namespace/sharding.xsd) for our next discussion.
> * [ ] Coding work to modify the configuration handling of `YAML`, `Java-based` for `ShardingJDBC` and `ShardingProxy`
> * [ ] Coding work to modify the configuration handling of `SpringNameSpace`, `SpringBoot` for `ShardingJDBC`
> * [ ] The similar changes for `ShardingSphere example module`
> * [ ] The API doc
>
> This is just a crude plan for this issue. However, you are the exact person responsible for it on-site, so follow your pace!
> One concern for me is that you had better make one PR **not so large**.
> If you need any help, please be free to ping me. :-)
@tristaZero
Hi Trista,
Here is the complete sample of YAML configuration file:
[config-sharding.txt](https://github.com/apache/shardingsphere/files/5288217/config-sharding.txt)
Please review and feedback.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 commented on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 commented on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-698185363
> Hi @johnny2002
> Your suggestion about `dataSourceTemplates` got some positive feedback from others.
>
> Before any programming, another important prerequisite is to design the configuration of [Spring NameSpace](https://github.com/apache/shardingsphere/blob/master/shardingsphere-features/shardingsphere-sharding/shardingsphere-sharding-spring/shardingsphere-sharding-spring-namespace/src/main/resources/META-INF/namespace/sharding.xsd).
>
> **Welcome your innovative idea**!
>
> BTW, are you interested in this issue？If so welcome your joining. Otherwise, I will make it involved in the community.
any progress?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 edited a comment on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 edited a comment on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-699625031
> @johnny2002 Sorry for my late reply.
>
> I assigned this issue to you, which means you are in charge of this feature, welcome!
> From my perspective, here are the brief procedure of this issue FYI.
>
> * [x] Design the configuration of `YAML`
> * [ ] Design the configuration API of [Spring NameSpace](https://github.com/apache/shardingsphere/blob/master/shardingsphere-features/shardingsphere-sharding/shardingsphere-sharding-spring/shardingsphere-sharding-spring-namespace/src/main/resources/META-INF/namespace/sharding.xsd) for our next discussion.
> * [ ] Coding work to modify the configuration handling of `YAML`, `Java-based` for `ShardingJDBC` and `ShardingProxy`
> * [ ] Coding work to modify the configuration handling of `SpringNameSpace`, `SpringBoot` for `ShardingJDBC`
> * [ ] The similar changes for `ShardingSphere example module`
> * [ ] The API doc
>
> This is just a crude plan for this issue. However, you are the exact person responsible for it on-site, so follow your pace!
> One concern for me is that you had better make one PR **not so large**.
> If you need any help, please be free to ping me. :-)
@tristaZero
Hi Trista,
Here is a complete sample of proposed YAML configuration file:
[config-sharding.txt](https://github.com/apache/shardingsphere/files/5288217/config-sharding.txt)
Since github doesn't support to upload yaml file, I renamed it to txt file.
Please review and feedback.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 commented on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 commented on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-693137537
In my suggestion, ${tablename} means logic table name.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 edited a comment on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 edited a comment on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-706857334
Hi Trista,
PR: https://github.com/apache/shardingsphere/pull/7751
Please review.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 edited a comment on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 edited a comment on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-699625031
> @johnny2002 Sorry for my late reply.
>
> I assigned this issue to you, which means you are in charge of this feature, welcome!
> From my perspective, here are the brief procedure of this issue FYI.
>
> * [x] Design the configuration of `YAML`
> * [ ] Design the configuration API of [Spring NameSpace](https://github.com/apache/shardingsphere/blob/master/shardingsphere-features/shardingsphere-sharding/shardingsphere-sharding-spring/shardingsphere-sharding-spring-namespace/src/main/resources/META-INF/namespace/sharding.xsd) for our next discussion.
> * [ ] Coding work to modify the configuration handling of `YAML`, `Java-based` for `ShardingJDBC` and `ShardingProxy`
> * [ ] Coding work to modify the configuration handling of `SpringNameSpace`, `SpringBoot` for `ShardingJDBC`
> * [ ] The similar changes for `ShardingSphere example module`
> * [ ] The API doc
>
> This is just a crude plan for this issue. However, you are the exact person responsible for it on-site, so follow your pace!
> One concern for me is that you had better make one PR **not so large**.
> If you need any help, please be free to ping me. :-)
Hi Trista,
Here is the complete sample of YAML configuration file:
[config-sharding.txt](https://github.com/apache/shardingsphere/files/5288217/config-sharding.txt)
Please review and feedback.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 commented on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 commented on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-693187128
> Hi @johnny2002
> `actual-data-nodes` expresses the positions of actual tables. In general, `logic_table_a` and `logic_table_b` have the same table rule though, the `actual-data-nodes` is different, right?
>
> BTW, `defaultTableRule` seems to fit your case. What do you think? Plus, we gave a huge improvement in the upcoming 5.x release configuration, like the following example.
>
> ```yaml
> schemaName: sharding_db
>
> dataSourceCommon:
> username: root
> password:
> connectionTimeoutMilliseconds: 30000
> idleTimeoutMilliseconds: 60000
> maxLifetimeMilliseconds: 1800000
> maxPoolSize: 50
> minPoolSize: 1
> maintenanceIntervalMilliseconds: 30000
>
> dataSources:
> ds_0:
> url: jdbc:mysql://127.0.0.1:3306/demo_ds_0?serverTimezone=UTC&useSSL=false
> ds_1:
> url: jdbc:mysql://127.0.0.1:3306/demo_ds_1?serverTimezone=UTC&useSSL=false
>
> rules:
> - !SHARDING
> tables:
> t_order:
> actualDataNodes: ds_${0..1}.t_order_${0..1}
> tableStrategy:
> standard:
> shardingColumn: order_id
> shardingAlgorithmName: t_order_inline
> keyGenerateStrategy:
> column: order_id
> keyGeneratorName: snowflake
> t_order_item:
> actualDataNodes: ds_${0..1}.t_order_item_${0..1}
> tableStrategy:
> standard:
> shardingColumn: order_id
> shardingAlgorithmName: t_order_item_inline
> keyGenerateStrategy:
> column: order_item_id
> keyGeneratorName: snowflake
> bindingTables:
> - t_order,t_order_item
> defaultDatabaseStrategy:
> standard:
> shardingColumn: user_id
> shardingAlgorithmName: database_inline
> defaultTableStrategy:
> none:
>
> shardingAlgorithms:
> database_inline:
> type: INLINE
> props:
> algorithm-expression: ds_${user_id % 2}
> t_order_inline:
> type: INLINE
> props:
> algorithm-expression: t_order_${order_id % 2}
> t_order_item_inline:
> type: INLINE
> props:
> algorithm-expression: t_order_item_${order_id % 2}
>
> keyGenerators:
> snowflake:
> type: SNOWFLAKE
> props:
> worker-id: 123
> ```
Hi Trista，
1. actual-data-nodes expresses the positions of actual tables. In general, logic_table_a and logic_table_b have the same table rule though, the actual-data-nodes is different, right?
【Exactly, Yes】
2. defaultTableRule seems to fit your case
【No，In most of cases, defaultTableRule apply to not-sharding tables】
In my real case, some tables sharding by column "customer_id", some tables sharding by date. For reference tables and parameter tables, no sharding.
Even some tables sharding with same column, while the sub-tables count may not be same.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] tristaZero commented on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
tristaZero commented on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-695787679
In some cases, it is indeed. On the other hand, if you want to use the same sharding algorithm many times, then a reference name is simpler, IMO.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 edited a comment on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 edited a comment on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-699725247
This is another way. By this way, we try to compatible with config of old version, and the same style of datasource template.
```yaml
rules:
- !SHARDING
tables:
template_item_id: #tables that sharding by item_id
actualDataNodes: ds_${0..1}.${LOGIC_TABLE}_${0..4}
databaseStrategy:
type: INLINE
shardingColumns: item_id
props:
#algorithm-expression: ds_${item_id % 2}
algorithm-expression: ${item_id % 2} #ignore "ds_", since it's already in actualDataNodes
tableStrategy:
type: INLINE
shardingColumns: item_id
props:
algorithm-expression: ${item_id % 5} #ignore "${LOGIC_TABLE}_", since it's already in actualDataNodes
template_order_id: #tables that sharding by order_id
actualDataNodes: ds_${0..1}.${LOGIC_TABLE}${0..9}
#use defaultDatabaseStrategy
tableStrategy:
type: INLINE
shardingColumns: order_id
props:
algorithm-expression: ${order_id % 10} #ignore "${LOGIC_TABLE}_", since it's already in actualDataNodes
t_order:
template: template_order_id
t_order_item:
template: template_order_id
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 edited a comment on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 edited a comment on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-699625031
> @johnny2002 Sorry for my late reply.
>
> I assigned this issue to you, which means you are in charge of this feature, welcome!
> From my perspective, here are the brief procedure of this issue FYI.
>
> * [x] Design the configuration of `YAML`
> * [ ] Design the configuration API of [Spring NameSpace](https://github.com/apache/shardingsphere/blob/master/shardingsphere-features/shardingsphere-sharding/shardingsphere-sharding-spring/shardingsphere-sharding-spring-namespace/src/main/resources/META-INF/namespace/sharding.xsd) for our next discussion.
> * [x] Coding work to modify the configuration handling of `YAML`, `Java-based` for `ShardingJDBC` and `ShardingProxy`
> * [ ] Coding work to modify the configuration handling of `SpringNameSpace`, `SpringBoot` for `ShardingJDBC`
> * [ ] The similar changes for `ShardingSphere example module`
> * [ ] The API doc
>
> This is just a crude plan for this issue. However, you are the exact person responsible for it on-site, so follow your pace!
> One concern for me is that you had better make one PR **not so large**.
> If you need any help, please be free to ping me. :-)
@tristaZero
Hi Trista,
Here is a complete sample of proposed YAML configuration file:
[config-sharding.txt](https://github.com/apache/shardingsphere/files/5288217/config-sharding.txt)
Since github doesn't support to upload yaml file, I renamed it to txt file.
I am working on another style of strategy config, will upload soon.
Please review and feedback.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 edited a comment on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 edited a comment on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-699625031
> @johnny2002 Sorry for my late reply.
>
> I assigned this issue to you, which means you are in charge of this feature, welcome!
> From my perspective, here are the brief procedure of this issue FYI.
>
> * [x] Design the configuration of `YAML`
> * [ ] Design the configuration API of [Spring NameSpace](https://github.com/apache/shardingsphere/blob/master/shardingsphere-features/shardingsphere-sharding/shardingsphere-sharding-spring/shardingsphere-sharding-spring-namespace/src/main/resources/META-INF/namespace/sharding.xsd) for our next discussion.
> * [x] Coding work to modify the configuration handling of `YAML`, `Java-based` for `ShardingJDBC` and `ShardingProxy`
> * [ ] Coding work to modify the configuration handling of `SpringNameSpace`, `SpringBoot` for `ShardingJDBC`
> * [ ] The similar changes for `ShardingSphere example module`
> * [x] The API doc
>
> This is just a crude plan for this issue. However, you are the exact person responsible for it on-site, so follow your pace!
> One concern for me is that you had better make one PR **not so large**.
> If you need any help, please be free to ping me. :-)
@tristaZero
Hi Trista,
Here is a complete sample of proposed YAML configuration file:
[config-sharding.txt](https://github.com/apache/shardingsphere/files/5288217/config-sharding.txt)
Since github doesn't support to upload yaml file, I renamed it to txt file.
I am working on another style of strategy config, will upload soon.
Please review and feedback.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 edited a comment on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 edited a comment on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-699625031
> @johnny2002 Sorry for my late reply.
>
> I assigned this issue to you, which means you are in charge of this feature, welcome!
> From my perspective, here are the brief procedure of this issue FYI.
>
> * [x] Design the configuration of `YAML`
> * [ ] Design the configuration API of [Spring NameSpace](https://github.com/apache/shardingsphere/blob/master/shardingsphere-features/shardingsphere-sharding/shardingsphere-sharding-spring/shardingsphere-sharding-spring-namespace/src/main/resources/META-INF/namespace/sharding.xsd) for our next discussion.
> * [ ] Coding work to modify the configuration handling of `YAML`, `Java-based` for `ShardingJDBC` and `ShardingProxy`
> * [ ] Coding work to modify the configuration handling of `SpringNameSpace`, `SpringBoot` for `ShardingJDBC`
> * [ ] The similar changes for `ShardingSphere example module`
> * [ ] The API doc
>
> This is just a crude plan for this issue. However, you are the exact person responsible for it on-site, so follow your pace!
> One concern for me is that you had better make one PR **not so large**.
> If you need any help, please be free to ping me. :-)
@tristaZero
Hi Trista,
Here is a complete sample of proposed YAML configuration file:
[config-sharding.txt](https://github.com/apache/shardingsphere/files/5288217/config-sharding.txt)
Since github doesn't support to upload yaml file, I renamed it to txt file.
I am working on another style of strategy config, will upload soon.
Please review and feedback.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] tristaZero commented on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
tristaZero commented on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-699755861
Hi @johnny2002,
Thanks for your detailed design doc. :)
Here are some of my thoughts. Welcome your review.
- `dataSourceTemplates`
This configuration seems clear, so I suppose we can first start with it. As you know, ShardingSphere is an extensive map needing more effort for you to explore so that we could start this journey by this work first. Otherwise, you have to face a considerable challenge. **One tip I need to emphasize** that I wish this item is optional, i.e., if there are no many data sources, users could configure the whole props for data sources one by one.
- `tables`-1
```yaml
rules:
- !SHARDING
shardingTables:
- #Sample1: all tables Sharding by item_id should be under this group
tables: t_order, t_order_item
```
I prefer to group tables having the same configuration! However, our 5.x release is coming soon, and it looks your changes will have a significant impact on the API. I am concerned that it will confuse users a lot. Hence, do you think we could have slightly smaller changes for them? But please take it easy. We could first consider the change for `dataSources` and put off this one later.
- `table`-1
```yaml
rules:
- !SHARDING
tables:
template_item_id:
t_order:
```
IMO, this example will confuse users, since `template_item_id ` differ from `t_order`, doesn't it?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] tristaZero commented on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
tristaZero commented on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-696553162
Hi @johnny2002
Your suggestion about `dataSourceTemplates` got some positive feedback from others.
Before any programming, another important prerequisite is to design the configuration of [Spring NameSpace](https://github.com/apache/shardingsphere/blob/master/shardingsphere-features/shardingsphere-sharding/shardingsphere-sharding-spring/shardingsphere-sharding-spring-namespace/src/main/resources/META-INF/namespace/sharding.xsd).
**Welcome your innovative idea**!
BTW, are you interested in this issue？If so welcome your joining. Otherwise, I will make it involved in the community.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 edited a comment on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 edited a comment on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-693187128
> Hi @johnny2002
> `actual-data-nodes` expresses the positions of actual tables. In general, `logic_table_a` and `logic_table_b` have the same table rule though, the `actual-data-nodes` is different, right?
>
> BTW, `defaultTableRule` seems to fit your case. What do you think? Plus, we gave a huge improvement in the upcoming 5.x release configuration, like the following example.
>
> ```yaml
> schemaName: sharding_db
>
> dataSourceCommon:
> username: root
> password:
> connectionTimeoutMilliseconds: 30000
> idleTimeoutMilliseconds: 60000
> maxLifetimeMilliseconds: 1800000
> maxPoolSize: 50
> minPoolSize: 1
> maintenanceIntervalMilliseconds: 30000
>
> dataSources:
> ds_0:
> url: jdbc:mysql://127.0.0.1:3306/demo_ds_0?serverTimezone=UTC&useSSL=false
> ds_1:
> url: jdbc:mysql://127.0.0.1:3306/demo_ds_1?serverTimezone=UTC&useSSL=false
>
> rules:
> - !SHARDING
> tables:
> t_order:
> actualDataNodes: ds_${0..1}.t_order_${0..1}
> tableStrategy:
> standard:
> shardingColumn: order_id
> shardingAlgorithmName: t_order_inline
> keyGenerateStrategy:
> column: order_id
> keyGeneratorName: snowflake
> t_order_item:
> actualDataNodes: ds_${0..1}.t_order_item_${0..1}
> tableStrategy:
> standard:
> shardingColumn: order_id
> shardingAlgorithmName: t_order_item_inline
> keyGenerateStrategy:
> column: order_item_id
> keyGeneratorName: snowflake
> bindingTables:
> - t_order,t_order_item
> defaultDatabaseStrategy:
> standard:
> shardingColumn: user_id
> shardingAlgorithmName: database_inline
> defaultTableStrategy:
> none:
>
> shardingAlgorithms:
> database_inline:
> type: INLINE
> props:
> algorithm-expression: ds_${user_id % 2}
> t_order_inline:
> type: INLINE
> props:
> algorithm-expression: t_order_${order_id % 2}
> t_order_item_inline:
> type: INLINE
> props:
> algorithm-expression: t_order_item_${order_id % 2}
>
> keyGenerators:
> snowflake:
> type: SNOWFLAKE
> props:
> worker-id: 123
> ```
Hi Trista，
1. actual-data-nodes expresses the positions of actual tables. In general, logic_table_a and logic_table_b have the same table rule though, the actual-data-nodes is different, right?
【Exactly, Yes】
2. defaultTableRule seems to fit your case
【No，In most of cases, defaultTableRule only has one, while I have some different sharding rules】
In my real case, some tables sharding by column "customer_id", some tables sharding by date. For reference tables and parameter tables, no sharding.
Even some tables sharding with same column, while the sub-tables count may not be same.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 commented on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 commented on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-693192702
I think, you should let developers to create multiple datasource templates. Sometimes, one app may need to connect to difference databases, for an example, business database in oracle and parameter database in mysql.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 commented on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 commented on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-699625031
> @johnny2002 Sorry for my late reply.
>
> I assigned this issue to you, which means you are in charge of this feature, welcome!
> From my perspective, here are the brief procedure of this issue FYI.
>
> * [x] Design the configuration of `YAML`
> * [ ] Design the configuration API of [Spring NameSpace](https://github.com/apache/shardingsphere/blob/master/shardingsphere-features/shardingsphere-sharding/shardingsphere-sharding-spring/shardingsphere-sharding-spring-namespace/src/main/resources/META-INF/namespace/sharding.xsd) for our next discussion.
> * [ ] Coding work to modify the configuration handling of `YAML`, `Java-based` for `ShardingJDBC` and `ShardingProxy`
> * [ ] Coding work to modify the configuration handling of `SpringNameSpace`, `SpringBoot` for `ShardingJDBC`
> * [ ] The similar changes for `ShardingSphere example module`
> * [ ] The API doc
>
> This is just a crude plan for this issue. However, you are the exact person responsible for it on-site, so follow your pace!
> One concern for me is that you had better make one PR **not so large**.
> If you need any help, please be free to ping me. :-)
Hi Trista,
Here is the complete sample of YAML configuration file:
[config-sharding.txt](https://github.com/apache/shardingsphere/files/5288217/config-sharding.txt)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 commented on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 commented on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-698185363
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 edited a comment on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 edited a comment on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-693187128
> Hi @johnny2002
> `actual-data-nodes` expresses the positions of actual tables. In general, `logic_table_a` and `logic_table_b` have the same table rule though, the `actual-data-nodes` is different, right?
>
> BTW, `defaultTableRule` seems to fit your case. What do you think? Plus, we gave a huge improvement in the upcoming 5.x release configuration, like the following example.
>
> ```yaml
> schemaName: sharding_db
>
> dataSourceCommon:
> username: root
> password:
> connectionTimeoutMilliseconds: 30000
> idleTimeoutMilliseconds: 60000
> maxLifetimeMilliseconds: 1800000
> maxPoolSize: 50
> minPoolSize: 1
> maintenanceIntervalMilliseconds: 30000
>
> dataSources:
> ds_0:
> url: jdbc:mysql://127.0.0.1:3306/demo_ds_0?serverTimezone=UTC&useSSL=false
> ds_1:
> url: jdbc:mysql://127.0.0.1:3306/demo_ds_1?serverTimezone=UTC&useSSL=false
>
> rules:
> - !SHARDING
> tables:
> t_order:
> actualDataNodes: ds_${0..1}.t_order_${0..1}
> tableStrategy:
> standard:
> shardingColumn: order_id
> shardingAlgorithmName: t_order_inline
> keyGenerateStrategy:
> column: order_id
> keyGeneratorName: snowflake
> t_order_item:
> actualDataNodes: ds_${0..1}.t_order_item_${0..1}
> tableStrategy:
> standard:
> shardingColumn: order_id
> shardingAlgorithmName: t_order_item_inline
> keyGenerateStrategy:
> column: order_item_id
> keyGeneratorName: snowflake
> bindingTables:
> - t_order,t_order_item
> defaultDatabaseStrategy:
> standard:
> shardingColumn: user_id
> shardingAlgorithmName: database_inline
> defaultTableStrategy:
> none:
>
> shardingAlgorithms:
> database_inline:
> type: INLINE
> props:
> algorithm-expression: ds_${user_id % 2}
> t_order_inline:
> type: INLINE
> props:
> algorithm-expression: t_order_${order_id % 2}
> t_order_item_inline:
> type: INLINE
> props:
> algorithm-expression: t_order_item_${order_id % 2}
>
> keyGenerators:
> snowflake:
> type: SNOWFLAKE
> props:
> worker-id: 123
> ```
Hi Trista，
1. actual-data-nodes expresses the positions of actual tables. In general, logic_table_a and logic_table_b have the same table rule though, the actual-data-nodes is different, right?
【Exactly, Yes】
2. defaultTableRule seems to fit your case
【No, defaultTableRule only has one, while I have some different sharding rules】
In my real case, some tables sharding by column "customer_id", some tables sharding by date. even sharding by same column, while they may have different slices. For some tables, like reference tables and parameter tables, no sharding.
Even some tables sharding with same column, while the sub-tables count may not be same.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] johnny2002 edited a comment on issue #7471: Simplify configuration file
Posted by GitBox <gi...@apache.org>.
johnny2002 edited a comment on issue #7471:
URL: https://github.com/apache/shardingsphere/issues/7471#issuecomment-699725247
```yaml
rules:
- !SHARDING
tables:
template_item_id: #tables that sharding by item_id
actualDataNodes: ds_${0..1}.${LOGIC_TABLE}_${0..4}
databaseStrategy:
type: INLINE
shardingColumns: item_id
props:
#algorithm-expression: ds_${item_id % 2}
algorithm-expression: ${item_id % 2} #ignore "ds_", since it's already in actualDataNodes
tableStrategy:
type: INLINE
shardingColumns: item_id
props:
algorithm-expression: ${item_id % 5} #ignore "${LOGIC_TABLE}_", since it's already in actualDataNodes
template_order_id: #tables that sharding by order_id
actualDataNodes: ds_${0..1}.${LOGIC_TABLE}${0..9}
#use defaultDatabaseStrategy
tableStrategy:
type: INLINE
shardingColumns: order_id
props:
algorithm-expression: ${order_id % 10} #ignore "${LOGIC_TABLE}_", since it's already in actualDataNodes
t_order:
template: template_order_id
t_order_item:
template: template_order_id
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org