You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2022/01/29 10:13:49 UTC

[GitHub] [shardingsphere] sandynz opened a new pull request #15190: Reuse sourceDataSource and sourceMetaDataLoader in scaling job process

sandynz opened a new pull request #15190:
URL: https://github.com/apache/shardingsphere/pull/15190


   Purpose:
   - Better performance of metadata loading
   - Cleaner code
   
   Changes proposed in this pull request:
   - Initialize sourceDataSource and sourceMetaDataLoader in job context
   - Reuse sourceDataSource and sourceMetaDataLoader in job process
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [shardingsphere] codecov-commenter commented on pull request #15190: Reuse sourceDataSource and sourceMetaDataLoader in scaling job process

Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #15190:
URL: https://github.com/apache/shardingsphere/pull/15190#issuecomment-1024996667


   # [Codecov](https://codecov.io/gh/apache/shardingsphere/pull/15190?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#15190](https://codecov.io/gh/apache/shardingsphere/pull/15190?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d6de959) into [master](https://codecov.io/gh/apache/shardingsphere/commit/12d002aa46c233b5a102dc841a986320b58097f7?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (12d002a) will **increase** coverage by `0.10%`.
   > The diff coverage is `17.39%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/shardingsphere/pull/15190/graphs/tree.svg?width=650&height=150&src=pr&token=ZvlXpWa7so&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/shardingsphere/pull/15190?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@             Coverage Diff              @@
   ##             master   #15190      +/-   ##
   ============================================
   + Coverage     59.78%   59.88%   +0.10%     
   - Complexity     1923     1924       +1     
   ============================================
     Files          3170     3172       +2     
     Lines         47482    47553      +71     
     Branches       8079     8092      +13     
   ============================================
   + Hits          28385    28477      +92     
   + Misses        16802    16761      -41     
   - Partials       2295     2315      +20     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/shardingsphere/pull/15190?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [.../core/ingest/dumper/AbstractIncrementalDumper.java](https://codecov.io/gh/apache/shardingsphere/pull/15190/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUta2VybmVsL3NoYXJkaW5nc3BoZXJlLWRhdGEtcGlwZWxpbmUvc2hhcmRpbmdzcGhlcmUtZGF0YS1waXBlbGluZS1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9zaGFyZGluZ3NwaGVyZS9kYXRhL3BpcGVsaW5lL2NvcmUvaW5nZXN0L2R1bXBlci9BYnN0cmFjdEluY3JlbWVudGFsRHVtcGVyLmphdmE=) | `0.00% <0.00%> (ø)` | |
   | [...ne/core/ingest/dumper/AbstractInventoryDumper.java](https://codecov.io/gh/apache/shardingsphere/pull/15190/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUta2VybmVsL3NoYXJkaW5nc3BoZXJlLWRhdGEtcGlwZWxpbmUvc2hhcmRpbmdzcGhlcmUtZGF0YS1waXBlbGluZS1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9zaGFyZGluZ3NwaGVyZS9kYXRhL3BpcGVsaW5lL2NvcmUvaW5nZXN0L2R1bXBlci9BYnN0cmFjdEludmVudG9yeUR1bXBlci5qYXZh) | `0.00% <0.00%> (ø)` | |
   | [...phere/data/pipeline/core/task/IncrementalTask.java](https://codecov.io/gh/apache/shardingsphere/pull/15190/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUta2VybmVsL3NoYXJkaW5nc3BoZXJlLWRhdGEtcGlwZWxpbmUvc2hhcmRpbmdzcGhlcmUtZGF0YS1waXBlbGluZS1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9zaGFyZGluZ3NwaGVyZS9kYXRhL3BpcGVsaW5lL2NvcmUvdGFzay9JbmNyZW1lbnRhbFRhc2suamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [...gsphere/data/pipeline/core/task/InventoryTask.java](https://codecov.io/gh/apache/shardingsphere/pull/15190/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUta2VybmVsL3NoYXJkaW5nc3BoZXJlLWRhdGEtcGlwZWxpbmUvc2hhcmRpbmdzcGhlcmUtZGF0YS1waXBlbGluZS1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9zaGFyZGluZ3NwaGVyZS9kYXRhL3BpcGVsaW5lL2NvcmUvdGFzay9JbnZlbnRvcnlUYXNrLmphdmE=) | `0.00% <0.00%> (ø)` | |
   | [...ne/scenario/rulealtered/RuleAlteredJobContext.java](https://codecov.io/gh/apache/shardingsphere/pull/15190/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUta2VybmVsL3NoYXJkaW5nc3BoZXJlLWRhdGEtcGlwZWxpbmUvc2hhcmRpbmdzcGhlcmUtZGF0YS1waXBlbGluZS1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9zaGFyZGluZ3NwaGVyZS9kYXRhL3BpcGVsaW5lL3NjZW5hcmlvL3J1bGVhbHRlcmVkL1J1bGVBbHRlcmVkSm9iQ29udGV4dC5qYXZh) | `0.00% <0.00%> (ø)` | |
   | [...e/scenario/rulealtered/RuleAlteredJobPreparer.java](https://codecov.io/gh/apache/shardingsphere/pull/15190/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUta2VybmVsL3NoYXJkaW5nc3BoZXJlLWRhdGEtcGlwZWxpbmUvc2hhcmRpbmdzcGhlcmUtZGF0YS1waXBlbGluZS1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9zaGFyZGluZ3NwaGVyZS9kYXRhL3BpcGVsaW5lL3NjZW5hcmlvL3J1bGVhbHRlcmVkL1J1bGVBbHRlcmVkSm9iUHJlcGFyZXIuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [...rio/rulealtered/prepare/InventoryTaskSplitter.java](https://codecov.io/gh/apache/shardingsphere/pull/15190/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUta2VybmVsL3NoYXJkaW5nc3BoZXJlLWRhdGEtcGlwZWxpbmUvc2hhcmRpbmdzcGhlcmUtZGF0YS1waXBlbGluZS1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9zaGFyZGluZ3NwaGVyZS9kYXRhL3BpcGVsaW5lL3NjZW5hcmlvL3J1bGVhbHRlcmVkL3ByZXBhcmUvSW52ZW50b3J5VGFza1NwbGl0dGVyLmphdmE=) | `0.00% <0.00%> (ø)` | |
   | [...gsphere/scaling/core/job/dumper/DumperFactory.java](https://codecov.io/gh/apache/shardingsphere/pull/15190/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUta2VybmVsL3NoYXJkaW5nc3BoZXJlLWRhdGEtcGlwZWxpbmUvc2hhcmRpbmdzcGhlcmUtZGF0YS1waXBlbGluZS1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9zaGFyZGluZ3NwaGVyZS9zY2FsaW5nL2NvcmUvam9iL2R1bXBlci9EdW1wZXJGYWN0b3J5LmphdmE=) | `0.00% <0.00%> (ø)` | |
   | [.../pipeline/opengauss/ingest/OpenGaussWalDumper.java](https://codecov.io/gh/apache/shardingsphere/pull/15190/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUta2VybmVsL3NoYXJkaW5nc3BoZXJlLWRhdGEtcGlwZWxpbmUvc2hhcmRpbmdzcGhlcmUtZGF0YS1waXBlbGluZS1kaWFsZWN0L3NoYXJkaW5nc3BoZXJlLWRhdGEtcGlwZWxpbmUtb3BlbmdhdXNzL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9zaGFyZGluZ3NwaGVyZS9kYXRhL3BpcGVsaW5lL29wZW5nYXVzcy9pbmdlc3QvT3BlbkdhdXNzV2FsRHVtcGVyLmphdmE=) | `0.00% <0.00%> (ø)` | |
   | [.../pipeline/mysql/ingest/MySQLIncrementalDumper.java](https://codecov.io/gh/apache/shardingsphere/pull/15190/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUta2VybmVsL3NoYXJkaW5nc3BoZXJlLWRhdGEtcGlwZWxpbmUvc2hhcmRpbmdzcGhlcmUtZGF0YS1waXBlbGluZS1kaWFsZWN0L3NoYXJkaW5nc3BoZXJlLWRhdGEtcGlwZWxpbmUtbXlzcWwvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NoYXJkaW5nc3BoZXJlL2RhdGEvcGlwZWxpbmUvbXlzcWwvaW5nZXN0L015U1FMSW5jcmVtZW50YWxEdW1wZXIuamF2YQ==) | `73.62% <100.00%> (ø)` | |
   | ... and [29 more](https://codecov.io/gh/apache/shardingsphere/pull/15190/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/shardingsphere/pull/15190?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/shardingsphere/pull/15190?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [12d002a...d6de959](https://codecov.io/gh/apache/shardingsphere/pull/15190?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [shardingsphere] terrymanu merged pull request #15190: Reuse sourceDataSource and sourceMetaDataLoader in scaling job process

Posted by GitBox <gi...@apache.org>.
terrymanu merged pull request #15190:
URL: https://github.com/apache/shardingsphere/pull/15190


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [shardingsphere] sandynz commented on pull request #15190: Reuse sourceDataSource and sourceMetaDataLoader in scaling job process

Posted by GitBox <gi...@apache.org>.
sandynz commented on pull request #15190:
URL: https://github.com/apache/shardingsphere/pull/15190#issuecomment-1024890357


   Real test on MySQL 5.7 passed on local.
   ```
   mysql> show scaling list;
   +-----------------+----------------------+----------------------+--------+---------------------+---------------------+
   | id              | tables               | sharding_total_count | active | create_time         | stop_time           |
   +-----------------+----------------------+----------------------+--------+---------------------+---------------------+
   | 496507791863449 | t_order,t_order_item | 2                    | false  | 2022-01-29 18:07:16 | 2022-01-29 18:09:00 |
   +-----------------+----------------------+----------------------+--------+---------------------+---------------------+
   1 row in set (0.02 sec)
   
   mysql> show scaling status 496507791863449;
   +------+-------------+----------+--------+-------------------------------+--------------------------+
   | item | data_source | status   | active | inventory_finished_percentage | incremental_idle_minutes |
   +------+-------------+----------+--------+-------------------------------+--------------------------+
   | 0    | ds_0        | FINISHED | false  | 100                           | 2                        |
   | 1    | ds_1        | FINISHED | false  | 100                           | 2                        |
   +------+-------------+----------+--------+-------------------------------+--------------------------+
   2 rows in set (0.02 sec)
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org