You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@shardingsphere.apache.org by sushixiang <sh...@qq.com> on 2020/04/29 05:35:13 UTC
[SHARDING-SCALING]Optimization about job start API
Hi everyone,
As we know, sharding-scaling can be used for migrating data from single datasource to ShardingSphere when first using ShardingSphere, and the start API is showed below:
curl -X POST \
http://localhost:8888/shardingscaling/job/start \
-H 'content-type: application/json' \
-d '{
"ruleConfiguration": {
"sourceDatasource": "ds_0: !!YamlDataSourceConfiguration\n dataSourceClassName: com.zaxxer.hikari.HikariDataSource\n properties:\n jdbcUrl: jdbc:mysql://127.0.0.1:3306/test?serverTimezone=UTC&useSSL=false\n username: root\n password: '\''123456'\''\n connectionTimeout: 30000\n idleTimeout: 60000\n maxLifetime: 1800000\n maxPoolSize: 50\n minPoolSize: 1\n maintenanceIntervalMilliseconds: 30000\n readOnly: false\n",
"sourceRule": "tables:\n t_order:\n actualDataNodes: ds_0.t_order\n tableStrategy:\n none",
"destinationDataSources": {
"name": "dt_0",
"password": "123456",
"url": "jdbc:mysql://127.0.0.1:3306/test2?serverTimezone=UTC&useSSL=false",
"username": "root"
}
},
"jobConfiguration": {
"concurrency": 3
}
}'
But I think the configuration may be much complex for the fisrt use, if we need migrate all tables from the source database, we should config all tables in the 'sourceRule' item, and for first use, we may not have the concept of table rules.
So, I think a simpler configuration may be needed, not only the first data migrate scenario, does anyone has some suggestions about it?
All data migrate scenarios can refer the link: https://shardingsphere.apache.org/document/current/en/features/sharding-scaling/core-features/
Re: [SHARDING-SCALING]Optimization about job start API
Posted by Sheng Wu <wu...@gmail.com>.
I think qq mail box and some mail local tool have this issue. Gmail showing
them.
Sheng Wu 吴晟
Twitter, wusheng1108
sushixiang <sh...@qq.com> 于2020年4月30日周四 上午8:58写道：
> Sorry, I don't know why the blanks turn into &nbsp;.
>
>
>
>
> ------------------ Original ------------------
> From: "KimmKing"<kimmking@apache.org>;
> Date: Wed, Apr 29, 2020 07:24 PM
> To: "dev"<dev@shardingsphere.apache.org>;"birdyanggr"<
> birdyanggr@163.com>;
>
> Subject: Re:[SHARDING-SCALING]Optimization about job start API
>
>
>
> a text format mail is better.
>
>
>
>
>
> At 2020-04-29 13:35:13, "sushixiang" <shixiangsu@qq.com> wrote:
> >Hi everyone,&nbsp;
> >As we know, sharding-scaling can be used for migrating data from
> single datasource to ShardingSphere when first using
> ShardingSphere,&nbsp;and the start API is showed below:
> >
> >
> >curl -X POST \
> >&nbsp; http://localhost:8888/shardingscaling/job/start \
> >&nbsp; -H 'content-type: application/json' \
> >&nbsp; -d '{
> >&nbsp; &nbsp;"ruleConfiguration": {
> >&nbsp; &nbsp; &nbsp; "sourceDatasource": "ds_0:
> !!YamlDataSourceConfiguration\n&nbsp; dataSourceClassName:
> com.zaxxer.hikari.HikariDataSource\n&nbsp; properties:\n&nbsp;
> &nbsp; jdbcUrl: jdbc:mysql://
> 127.0.0.1:3306/test?serverTimezone=UTC&amp;useSSL=false\n&nbsp
> <http://127.0.0.1:3306/test?serverTimezone=UTC&amp;useSSL=false%5Cn&nbsp>;
> &nbsp; username: root\n&nbsp; &nbsp; password:
> '\''123456'\''\n&nbsp; &nbsp; connectionTimeout: 30000\n&nbsp;
> &nbsp; idleTimeout: 60000\n&nbsp; &nbsp; maxLifetime:
> 1800000\n&nbsp; &nbsp; maxPoolSize: 50\n&nbsp; &nbsp;
> minPoolSize: 1\n&nbsp; &nbsp; maintenanceIntervalMilliseconds:
> 30000\n&nbsp; &nbsp; readOnly: false\n",
> >&nbsp; &nbsp; &nbsp; "sourceRule": "tables:\n&nbsp;
> t_order:\n&nbsp; &nbsp; actualDataNodes: ds_0.t_order\n&nbsp;
> &nbsp; tableStrategy:\n&nbsp; &nbsp; &nbsp; none",
> >&nbsp; &nbsp; &nbsp; "destinationDataSources": {
> >&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;"name": "dt_0",
> >&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;"password":
> "123456",
> >&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;"url":
> "jdbc:mysql://127.0.0.1:3306/test2?serverTimezone=UTC&amp;useSSL=false
> ",
> >&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;"username":
> "root"
> >&nbsp; &nbsp; &nbsp; }
> >&nbsp; &nbsp;},
> >&nbsp; &nbsp;"jobConfiguration": {
> >&nbsp; &nbsp; &nbsp; "concurrency": 3
> >&nbsp; &nbsp;}
> >}'
> >
> >
> >
> >But I think the configuration may be much complex for the fisrt use,
> if we need migrate all tables from the source database, we should config
> all tables in the 'sourceRule' item, and for first use, we may not have the
> concept of table rules.
> >
> >
> >So, I think a simpler configuration may be needed, not only the first
> data migrate scenario, does anyone has some suggestions about it?
> >
> >
> >All data migrate scenarios can refer the link:&nbsp;
> https://shardingsphere.apache.org/document/current/en/features/sharding-scaling/core-features/
Re:[SHARDING-SCALING]Optimization about job start API
Posted by sushixiang <sh...@qq.com>.
Sorry, I don't know why the blanks turn into &nbsp;.
------------------ Original ------------------
From: "KimmKing"<kimmking@apache.org>;
Date: Wed, Apr 29, 2020 07:24 PM
To: "dev"<dev@shardingsphere.apache.org>;"birdyanggr"<birdyanggr@163.com>;
Subject: Re:[SHARDING-SCALING]Optimization about job start API
a text format mail is better.
At 2020-04-29 13:35:13, "sushixiang" <shixiangsu@qq.com> wrote:
>Hi everyone,&nbsp;
>As we know, sharding-scaling can be used for migrating data from single datasource to ShardingSphere when first using ShardingSphere,&nbsp;and the start API is showed below:
>
>
>curl -X POST \
>&nbsp; http://localhost:8888/shardingscaling/job/start \
>&nbsp; -H 'content-type: application/json' \
>&nbsp; -d '{
>&nbsp; &nbsp;"ruleConfiguration": {
>&nbsp; &nbsp; &nbsp; "sourceDatasource": "ds_0: !!YamlDataSourceConfiguration\n&nbsp; dataSourceClassName: com.zaxxer.hikari.HikariDataSource\n&nbsp; properties:\n&nbsp; &nbsp; jdbcUrl: jdbc:mysql://127.0.0.1:3306/test?serverTimezone=UTC&amp;useSSL=false\n&nbsp; &nbsp; username: root\n&nbsp; &nbsp; password: '\''123456'\''\n&nbsp; &nbsp; connectionTimeout: 30000\n&nbsp; &nbsp; idleTimeout: 60000\n&nbsp; &nbsp; maxLifetime: 1800000\n&nbsp; &nbsp; maxPoolSize: 50\n&nbsp; &nbsp; minPoolSize: 1\n&nbsp; &nbsp; maintenanceIntervalMilliseconds: 30000\n&nbsp; &nbsp; readOnly: false\n",
>&nbsp; &nbsp; &nbsp; "sourceRule": "tables:\n&nbsp; t_order:\n&nbsp; &nbsp; actualDataNodes: ds_0.t_order\n&nbsp; &nbsp; tableStrategy:\n&nbsp; &nbsp; &nbsp; none",
>&nbsp; &nbsp; &nbsp; "destinationDataSources": {
>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;"name": "dt_0",
>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;"password": "123456",
>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;"url": "jdbc:mysql://127.0.0.1:3306/test2?serverTimezone=UTC&amp;useSSL=false",
>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;"username": "root"
>&nbsp; &nbsp; &nbsp; }
>&nbsp; &nbsp;},
>&nbsp; &nbsp;"jobConfiguration": {
>&nbsp; &nbsp; &nbsp; "concurrency": 3
>&nbsp; &nbsp;}
>}'
>
>
>
>But I think the configuration may be much complex for the fisrt use, if we need migrate all tables from the source database, we should config all tables in the 'sourceRule' item, and for first use, we may not have the concept of table rules.
>
>
>So, I think a simpler configuration may be needed, not only the first data migrate scenario, does anyone has some suggestions about it?
>
>
>All data migrate scenarios can refer the link:&nbsp;https://shardingsphere.apache.org/document/current/en/features/sharding-scaling/core-features/
Re:[SHARDING-SCALING]Optimization about job start API
Posted by KimmKing <ki...@apache.org>.
a text format mail is better.
At 2020-04-29 13:35:13, "sushixiang" <sh...@qq.com> wrote:
>Hi everyone,
>As we know, sharding-scaling can be used for migrating data from single datasource to ShardingSphere when first using ShardingSphere, and the start API is showed below:
>
>
>curl -X POST \
> http://localhost:8888/shardingscaling/job/start \
> -H 'content-type: application/json' \
> -d '{
> "ruleConfiguration": {
> "sourceDatasource": "ds_0: !!YamlDataSourceConfiguration\n dataSourceClassName: com.zaxxer.hikari.HikariDataSource\n properties:\n jdbcUrl: jdbc:mysql://127.0.0.1:3306/test?serverTimezone=UTC&useSSL=false\n username: root\n password: '\''123456'\''\n connectionTimeout: 30000\n idleTimeout: 60000\n maxLifetime: 1800000\n maxPoolSize: 50\n minPoolSize: 1\n maintenanceIntervalMilliseconds: 30000\n readOnly: false\n",
> "sourceRule": "tables:\n t_order:\n actualDataNodes: ds_0.t_order\n tableStrategy:\n none",
> "destinationDataSources": {
> "name": "dt_0",
> "password": "123456",
> "url": "jdbc:mysql://127.0.0.1:3306/test2?serverTimezone=UTC&useSSL=false",
> "username": "root"
> }
> },
> "jobConfiguration": {
> "concurrency": 3
> }
>}'
>
>
>
>But I think the configuration may be much complex for the fisrt use, if we need migrate all tables from the source database, we should config all tables in the 'sourceRule' item, and for first use, we may not have the concept of table rules.
>
>
>So, I think a simpler configuration may be needed, not only the first data migrate scenario, does anyone has some suggestions about it?
>
>
>All data migrate scenarios can refer the link: https://shardingsphere.apache.org/document/current/en/features/sharding-scaling/core-features/
Re:[SHARDING-SCALING]Optimization about job start API
Posted by Juan Pan <pa...@apache.org>.
Hi shixiang,
I am glad to see your discussion email,
which surely contains your thinking and opinion in dev list.
If you have any idea to express and welcome others' thoughts,
my suggestion is to put them in GitHub and open a discussion thread in dev list,
which helps present yourself through images or links, anyway.
Best,
Trista
Juan Pan (Trista)
Senior DBA & PPMC of Apache ShardingSphere(Incubating)
E-mail: panjuan@apache.org
On 04/29/2020 13:35，sushixiang<sh...@qq.com> wrote：
Hi everyone,
As we know, sharding-scaling can be used for migrating data from single datasource to ShardingSphere when first using ShardingSphere, and the start API is showed below:
curl -X POST \
http://localhost:8888/shardingscaling/job/start \
-H 'content-type: application/json' \
-d '{
"ruleConfiguration": {
"sourceDatasource": "ds_0: !!YamlDataSourceConfiguration\n dataSourceClassName: com.zaxxer.hikari.HikariDataSource\n properties:\n jdbcUrl: jdbc:mysql://127.0.0.1:3306/test?serverTimezone=UTC&useSSL=false\n username: root\n password: '\''123456'\''\n connectionTimeout: 30000\n idleTimeout: 60000\n maxLifetime: 1800000\n maxPoolSize: 50\n minPoolSize: 1\n maintenanceIntervalMilliseconds: 30000\n readOnly: false\n",
"sourceRule": "tables:\n t_order:\n actualDataNodes: ds_0.t_order\n tableStrategy:\n none",
"destinationDataSources": {
"name": "dt_0",
"password": "123456",
"url": "jdbc:mysql://127.0.0.1:3306/test2?serverTimezone=UTC&useSSL=false",
"username": "root"
}
},
"jobConfiguration": {
"concurrency": 3
}
}'
But I think the configuration may be much complex for the fisrt use, if we need migrate all tables from the source database, we should config all tables in the 'sourceRule' item, and for first use, we may not have the concept of table rules.
So, I think a simpler configuration may be needed, not only the first data migrate scenario, does anyone has some suggestions about it?
All data migrate scenarios can refer the link: https://shardingsphere.apache.org/document/current/en/features/sharding-scaling/core-features/