You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2021/12/10 13:24:09 UTC
[GitHub] [shardingsphere] sandynz opened a new pull request #14045: Separate sharding related code from scaling-core module part 3 and refactor job configuration structure
sandynz opened a new pull request #14045:
URL: https://github.com/apache/shardingsphere/pull/14045
Fixes #13875 part 8.
Changes proposed in this pull request:
- Remove SnowflakeKeyGenerateAlgorithm dependency from JobConfiguration, which is in `sharding-core`
- Add getActualDataSourceConfig for ShardingSphereJDBCDataSourceConfiguration, prepare for DataSourcePreparer refactoring
- Add JobDataNodeLine
- Add tablesFirstDataNodes in job handleConfig, prepare for DataSourcePreparer refactoring
- Refactor fields in job handleConfig
- Refactor DataSourcePreparer, separate sharding related code from scaling-core
- Refactor jobShardingDataNodes type in handleConfig
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] sandynz commented on pull request #14045: Separate sharding related code from scaling-core module part 3 and refactor job configuration structure
Posted by GitBox <gi...@apache.org>.
sandynz commented on pull request #14045:
URL: https://github.com/apache/shardingsphere/pull/14045#issuecomment-991018311
There's one CI run failed, still embed ZK related:
```
[INFO ] 2021-12-10 13:43:48.944 [main] o.a.s.s.c.fixture.EmbedTestingServer - Embed zookeeper server already exists 1, on 127.0.0.1:3181
[INFO ] 2021-12-10 13:43:49.132 [main] org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=localhost:3181 sessionTimeout=60000 watcher=org.apache.curator.ConnectionState@42504d1
[WARN ] 2021-12-10 13:43:51.315 [SyncThread:0] o.a.z.server.persistence.FileTxnLog - fsync-ing the write ahead log in SyncThread:0 took 2176ms which will adversely effect operation latency.File size is 67108880 bytes. See the ZooKeeper troubleshooting guide
[INFO ] 2021-12-10 13:43:51.631 [main] org.apache.zookeeper.ZooKeeper - Session: 0x100001307ce0001 closed
Error: Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.716 s <<< FAILURE! - in org.apache.shardingsphere.scaling.core.job.ScalingJobTest
Error: assertExecute(org.apache.shardingsphere.scaling.core.job.ScalingJobTest) Time elapsed: 2.532 s <<< ERROR!
org.apache.shardingsphere.mode.repository.cluster.ClusterPersistRepositoryException: org.apache.zookeeper.KeeperException$OperationTimeoutException: KeeperErrorCode = OperationTimeout
at org.apache.shardingsphere.scaling.core.job.ScalingJobTest.assertExecute(ScalingJobTest.java:56)
Caused by: org.apache.zookeeper.KeeperException$OperationTimeoutException: KeeperErrorCode = OperationTimeout
at org.apache.shardingsphere.scaling.core.job.ScalingJobTest.assertExecute(ScalingJobTest.java:56)
```
It might be caused by `fsync-ing`, it cost 2176ms, more than operation timeout.
Plan to implement Memory mode registry center implementation and use it for unit test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] codecov-commenter commented on pull request #14045: Separate sharding related code from scaling-core module part 3 and refactor job configuration structure
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #14045:
URL: https://github.com/apache/shardingsphere/pull/14045#issuecomment-991134881
# [Codecov](https://codecov.io/gh/apache/shardingsphere/pull/14045?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#14045](https://codecov.io/gh/apache/shardingsphere/pull/14045?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (73e71a7) into [master](https://codecov.io/gh/apache/shardingsphere/commit/4f01e866431564a5b52c154ec5b0da0690ddc801?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (4f01e86) will **decrease** coverage by `0.05%`.
> The diff coverage is `56.36%`.
[](https://codecov.io/gh/apache/shardingsphere/pull/14045?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #14045 +/- ##
============================================
- Coverage 63.48% 63.43% -0.06%
- Complexity 1771 1780 +9
============================================
Files 2810 2819 +9
Lines 42027 42150 +123
Branches 7172 7195 +23
============================================
+ Hits 26682 26736 +54
- Misses 13303 13359 +56
- Partials 2042 2055 +13
```
| [Impacted Files](https://codecov.io/gh/apache/shardingsphere/pull/14045?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ped/ShardingSphereJDBCDataSourceConfiguration.java](https://codecov.io/gh/apache/shardingsphere/pull/14045/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtaW5mcmEvc2hhcmRpbmdzcGhlcmUtaW5mcmEtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9zaGFyZGluZ3NwaGVyZS9pbmZyYS9jb25maWcvZGF0YXNvdXJjZS90eXBlZC9TaGFyZGluZ1NwaGVyZUpEQkNEYXRhU291cmNlQ29uZmlndXJhdGlvbi5qYXZh) | `51.85% <0.00%> (-6.49%)` | :arrow_down: |
| [...datasource/typed/TypedDataSourceConfiguration.java](https://codecov.io/gh/apache/shardingsphere/pull/14045/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtaW5mcmEvc2hhcmRpbmdzcGhlcmUtaW5mcmEtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9zaGFyZGluZ3NwaGVyZS9pbmZyYS9jb25maWcvZGF0YXNvdXJjZS90eXBlZC9UeXBlZERhdGFTb3VyY2VDb25maWd1cmF0aW9uLmphdmE=) | `16.66% <0.00%> (-10.61%)` | :arrow_down: |
| [...ta/pipeline/core/datasource/DataSourceManager.java](https://codecov.io/gh/apache/shardingsphere/pull/14045/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUta2VybmVsL3NoYXJkaW5nc3BoZXJlLWRhdGEtcGlwZWxpbmUvc2hhcmRpbmdzcGhlcmUtZGF0YS1waXBlbGluZS1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9zaGFyZGluZ3NwaGVyZS9kYXRhL3BpcGVsaW5lL2NvcmUvZGF0YXNvdXJjZS9EYXRhU291cmNlTWFuYWdlci5qYXZh) | `0.00% <ø> (ø)` | |
| [...dingsphere/migration/common/api/ScalingWorker.java](https://codecov.io/gh/apache/shardingsphere/pull/14045/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtc2NhbGluZy9zaGFyZGluZ3NwaGVyZS1zY2FsaW5nLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NoYXJkaW5nc3BoZXJlL21pZ3JhdGlvbi9jb21tb24vYXBpL1NjYWxpbmdXb3JrZXIuamF2YQ==) | `0.00% <ø> (ø)` | |
| [...ommon/job/preparer/AbstractDataSourcePreparer.java](https://codecov.io/gh/apache/shardingsphere/pull/14045/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtc2NhbGluZy9zaGFyZGluZ3NwaGVyZS1zY2FsaW5nLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NoYXJkaW5nc3BoZXJlL21pZ3JhdGlvbi9jb21tb24vam9iL3ByZXBhcmVyL0Fic3RyYWN0RGF0YVNvdXJjZVByZXBhcmVyLmphdmE=) | `54.54% <0.00%> (+21.21%)` | :arrow_up: |
| [.../check/consistency/DataConsistencyCheckerImpl.java](https://codecov.io/gh/apache/shardingsphere/pull/14045/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtc2NhbGluZy9zaGFyZGluZ3NwaGVyZS1zY2FsaW5nLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NoYXJkaW5nc3BoZXJlL3NjYWxpbmcvY29yZS9qb2IvY2hlY2svY29uc2lzdGVuY3kvRGF0YUNvbnNpc3RlbmN5Q2hlY2tlckltcGwuamF2YQ==) | `85.91% <ø> (ø)` | |
| [...rdingsphere/scaling/core/util/ScalingTaskUtil.java](https://codecov.io/gh/apache/shardingsphere/pull/14045/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtc2NhbGluZy9zaGFyZGluZ3NwaGVyZS1zY2FsaW5nLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NoYXJkaW5nc3BoZXJlL3NjYWxpbmcvY29yZS91dGlsL1NjYWxpbmdUYXNrVXRpbC5qYXZh) | `55.00% <0.00%> (+45.00%)` | :arrow_up: |
| [...component/checker/OpenGaussDataSourcePreparer.java](https://codecov.io/gh/apache/shardingsphere/pull/14045/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtc2NhbGluZy9zaGFyZGluZ3NwaGVyZS1zY2FsaW5nLWRpYWxlY3Qvc2hhcmRpbmdzcGhlcmUtc2NhbGluZy1vcGVuZ2F1c3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NoYXJkaW5nc3BoZXJlL3NjYWxpbmcvb3BlbmdhdXNzL2NvbXBvbmVudC9jaGVja2VyL09wZW5HYXVzc0RhdGFTb3VyY2VQcmVwYXJlci5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...ngsphere/scaling/core/config/JobConfiguration.java](https://codecov.io/gh/apache/shardingsphere/pull/14045/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtc2NhbGluZy9zaGFyZGluZ3NwaGVyZS1zY2FsaW5nLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NoYXJkaW5nc3BoZXJlL3NjYWxpbmcvY29yZS9jb25maWcvSm9iQ29uZmlndXJhdGlvbi5qYXZh) | `43.75% <16.66%> (-46.88%)` | :arrow_down: |
| [...schedule/ShardingRuleJobConfigurationPreparer.java](https://codecov.io/gh/apache/shardingsphere/pull/14045/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtc2NhbGluZy9zaGFyZGluZ3NwaGVyZS1zY2FsaW5nLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NoYXJkaW5nc3BoZXJlL3NoYXJkaW5nL3NjaGVkdWxlL1NoYXJkaW5nUnVsZUpvYkNvbmZpZ3VyYXRpb25QcmVwYXJlci5qYXZh) | `58.08% <31.81%> (-23.26%)` | :arrow_down: |
| ... and [63 more](https://codecov.io/gh/apache/shardingsphere/pull/14045/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/shardingsphere/pull/14045?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/shardingsphere/pull/14045?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [4f01e86...73e71a7](https://codecov.io/gh/apache/shardingsphere/pull/14045?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] tristaZero merged pull request #14045: Separate sharding related code from scaling-core module part 3 and refactor job configuration structure
Posted by GitBox <gi...@apache.org>.
tristaZero merged pull request #14045:
URL: https://github.com/apache/shardingsphere/pull/14045
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] tristaZero merged pull request #14045: Separate sharding related code from scaling-core module part 3 and refactor job configuration structure
Posted by GitBox <gi...@apache.org>.
tristaZero merged pull request #14045:
URL: https://github.com/apache/shardingsphere/pull/14045
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org