You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2022/01/10 08:00:20 UTC
[GitHub] [shardingsphere] RaigorJiang opened a new issue #14661: [DistSQL] Add some new syntax to manage scaling configurations.
RaigorJiang opened a new issue #14661:
URL: https://github.com/apache/shardingsphere/issues/14661
Hi community,
In order to facilitate users to manage the scaling configuration, we will add some new DistSQL syntax.
At the same time, since the scaling feature has been refactored to support different levels of configurations (#13875), DistSQL will also support different levels:
#### Example
1. Automatic scaling process, specifying minimal configuration
```sql
CREATE SHARDING SCALING scaling_name(
COMPLETION_DETECTOR(TYPE(NAME=IDLE, PROPERTIES("incremental-task-idle-minute-threshold"=30))),
DATA_CONSISTENCY_CHECKER(TYPE(NAME=DATA_MATCH, PROPERTIES("chunk-size"=1000)))
);
```
2. Automatic scaling process, specifying complete configuration
```sql
CREATE SHARDING SCALING scaling_name(
INPUT(
WORKER_THREAD=40,
BATCH_SIZE=1000,
RATE_LIMITER(TYPE(NAME=QPS, PROPERTIES("qps"=50)))
),
OUTPUT(
WORKER_THREAD=40,
BATCH_SIZE=1000,
RATE_LIMITER(TYPE(NAME=TPS, PROPERTIES("tps"=2000)))
),
STREAM_CHANNEL(TYPE(NAME=MEMORY, PROPERTIES("block-queue-size"=10000))),
COMPLETION_DETECTOR(TYPE(NAME=IDLE, PROPERTIES("incremental-task-idle-minute-threshold"=30))),
DATA_CONSISTENCY_CHECKER(TYPE(NAME=DATA_MATCH, PROPERTIES("chunk-size"=1000)))
);
```
3. Manual scaling process
```sql
CREATE SHARDING SCALING scaling_name(
INPUT(
WORKER_THREAD=40,
BATCH_SIZE=1000,
RATE_LIMITER(TYPE(NAME=QPS, PROPERTIES("qps"=50)))
),
OUTPUT(
WORKER_THREAD=40,
BATCH_SIZE=1000,
RATE_LIMITER(TYPE(NAME=TPS, PROPERTIES("tps"=2000)))
),
STREAM_CHANNEL(TYPE(NAME=MEMORY, PROPERTIES("block-queue-size"=10000)))
);
```
4. Empty configuration (done in #14551)
```sql
CREATE SHARDING SCALING scaling_name;
```
5. Drop scaling configuration (done in #14551)
```sql
DROP SHARDING SCALING scaling_name;
```
6. Enable scaling configuration
```sql
ENABLE SHARDING SCALING scaling_name;
```
7. Disable scaling configuration
```sql
DISABLE SHARDING SCALING scaling_name;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] sandynz commented on issue #14661: [DistSQL] Add some new syntax to manage scaling configurations.
Posted by GitBox <gi...@apache.org>.
sandynz commented on issue #14661:
URL: https://github.com/apache/shardingsphere/issues/14661#issuecomment-1021800032
> @sandynz Ok, after the release of 5.1.0, we adjusted all input params to be optional.
Thank's OK, thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] sandynz commented on issue #14661: [DistSQL] Add some new syntax to manage scaling configurations.
Posted by GitBox <gi...@apache.org>.
sandynz commented on issue #14661:
URL: https://github.com/apache/shardingsphere/issues/14661#issuecomment-1011790808
The design and implementation cost a short time, since scaling require DistSQL in `5.1.0` release recently. Thank you very much! @RaigorJiang
I've tested all the `CREATE` examples above, every one of them works. I just did more test now, and I found some situation:
1. Looks there're strict constraints.
Empty scaling configuration is allowed, it means `INPUT`, `OUTPUT`, `STREAM_CHANNEL`, `COMPLETION_DETECTOR` and `DATA_CONSISTENCY_CHECKER` are optional.
Seems they're not optional for now, in `minimal automatic process`, `complete automatic process` and `manual process`.
When any of them are not configured, it show error message. e.g.
```
mysql> CREATE SHARDING SCALING scaling_name1(
-> COMPLETION_DETECTOR(TYPE(NAME=IDLE, PROPERTIES("incremental-task-idle-minute-threshold"=30)))
-> );
ERROR 1235 (42000): This version of ShardingSphere-Proxy doesn't yet support this SQL. 'You have an error in your SQL syntax'
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] terrymanu closed issue #14661: [DistSQL] Add some new syntax to manage scaling configurations.
Posted by GitBox <gi...@apache.org>.
terrymanu closed issue #14661:
URL: https://github.com/apache/shardingsphere/issues/14661
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] RaigorJiang commented on issue #14661: [DistSQL] Add some new syntax to manage scaling configurations.
Posted by GitBox <gi...@apache.org>.
RaigorJiang commented on issue #14661:
URL: https://github.com/apache/shardingsphere/issues/14661#issuecomment-1019717273
@sandynz
Ok, after the release of 5.1.0, we adjusted all input params to be optional.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org