You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2022/09/14 01:49:27 UTC
[GitHub] [shardingsphere] azexcy opened a new pull request, #20962: Improve drop pipeline process and IT
azexcy opened a new pull request, #20962:
URL: https://github.com/apache/shardingsphere/pull/20962
Changes proposed in this pull request:
- Improve drop pipeline process and IT
---
Before committing this PR, I'm sure that I have checked the following options:
- [x] My code follows the [code of conduct](https://shardingsphere.apache.org/community/en/involved/conduct/code/) of this project.
- [x] I have self-reviewed the commit code.
- [x] I have triggered maven check: `mvn clean install -B -T2C -DskipTests -Dmaven.javadoc.skip=true -e`.
- [x] I have made corresponding changes to the documentation.
- [x] I have added corresponding unit tests for my changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] sandynz commented on a diff in pull request #20962: Improve drop pipeline process and IT
Posted by GitBox <gi...@apache.org>.
sandynz commented on code in PR #20962:
URL: https://github.com/apache/shardingsphere/pull/20962#discussion_r970230365
##########
shardingsphere-kernel/shardingsphere-data-pipeline/shardingsphere-data-pipeline-core/src/main/java/org/apache/shardingsphere/data/pipeline/core/api/impl/AbstractPipelineJobAPIImpl.java:
##########
@@ -75,7 +76,9 @@ public abstract class AbstractPipelineJobAPIImpl implements PipelineJobAPI {
@Override
public void createProcessConfiguration(final PipelineProcessConfiguration processConfig) {
PipelineProcessConfiguration existingProcessConfig = processConfigPersistService.load(getJobType());
- ShardingSpherePreconditions.checkState(null == existingProcessConfig, new CreateExistsProcessConfigurationException());
+ if (null != existingProcessConfig && ObjectUtils.allNotNull(existingProcessConfig.getRead(), existingProcessConfig.getWrite(), existingProcessConfig.getStreamChannel())) {
+ throw new CreateExistsProcessConfigurationException();
+ }
Review Comment:
1, Why add `ObjectUtils.allNotNull`, if there's part of process configured, create process configuration is still allowed?
2, Could we keep `ShardingSpherePreconditions.checkState`?
##########
shardingsphere-test/shardingsphere-integration-test/shardingsphere-integration-test-scaling/src/test/java/org/apache/shardingsphere/integration/data/pipeline/cases/migration/AbstractMigrationITCase.java:
##########
@@ -126,15 +126,14 @@ protected void startMigrationOrderItem(final boolean withSchema) throws SQLExcep
}
protected void addMigrationProcessConfig() throws SQLException {
- try {
- proxyExecuteWithLog(migrationDistSQLCommand.getAddMigrationProcessConfig(), 0);
- } catch (final SQLException ex) {
- if ("58000".equals(ex.getSQLState()) || "HY000".equals(ex.getSQLState())) {
- log.warn(ex.getMessage());
- return;
+ if (ENV.getItEnvType() == ITEnvTypeEnum.NATIVE) {
+ try {
+ proxyExecuteWithLog("DROP MIGRATION PROCESS CONFIGURATION '/read'", 1);
+ } catch (final SQLException ex) {
Review Comment:
`'/read'` could just clean `READ` configuraion, `'/'` could be used to clean process configuration
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] sandynz commented on a diff in pull request #20962: Improve drop pipeline process and IT
Posted by GitBox <gi...@apache.org>.
sandynz commented on code in PR #20962:
URL: https://github.com/apache/shardingsphere/pull/20962#discussion_r970258313
##########
shardingsphere-kernel/shardingsphere-data-pipeline/shardingsphere-data-pipeline-core/src/main/java/org/apache/shardingsphere/data/pipeline/core/api/impl/PipelineProcessConfigurationPersistService.java:
##########
@@ -44,7 +45,10 @@ public PipelineProcessConfiguration load(final JobType jobType) {
@Override
public void persist(final JobType jobType, final PipelineProcessConfiguration processConfig) {
- String yamlText = YamlEngine.marshal(PROCESS_CONFIG_SWAPPER.swapToYamlConfiguration(processConfig));
+ String yamlText = "";
+ if (ObjectUtils.anyNotNull(processConfig.getRead(), processConfig.getWrite(), processConfig.getStreamChannel())) {
+ yamlText = YamlEngine.marshal(PROCESS_CONFIG_SWAPPER.swapToYamlConfiguration(processConfig));
+ }
Review Comment:
Could we add `drop/delete` method in `PipelineMetaDataPersistService`? So it could be used for `DROP` DistSQL (SOURCE RESOURCE and PROCESS CONFIGURATION).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] sandynz commented on a diff in pull request #20962: Improve drop pipeline process and IT
Posted by GitBox <gi...@apache.org>.
sandynz commented on code in PR #20962:
URL: https://github.com/apache/shardingsphere/pull/20962#discussion_r970267485
##########
shardingsphere-kernel/shardingsphere-data-pipeline/shardingsphere-data-pipeline-core/src/main/java/org/apache/shardingsphere/data/pipeline/core/api/impl/PipelineProcessConfigurationPersistService.java:
##########
@@ -44,7 +45,10 @@ public PipelineProcessConfiguration load(final JobType jobType) {
@Override
public void persist(final JobType jobType, final PipelineProcessConfiguration processConfig) {
- String yamlText = YamlEngine.marshal(PROCESS_CONFIG_SWAPPER.swapToYamlConfiguration(processConfig));
+ String yamlText = "";
+ if (ObjectUtils.anyNotNull(processConfig.getRead(), processConfig.getWrite(), processConfig.getStreamChannel())) {
+ yamlText = YamlEngine.marshal(PROCESS_CONFIG_SWAPPER.swapToYamlConfiguration(processConfig));
+ }
Review Comment:
Or we could keep the persist logic, and check whether it's empty or not in `load` method
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] sandynz merged pull request #20962: Improve drop pipeline process configuration and IT
Posted by GitBox <gi...@apache.org>.
sandynz merged PR #20962:
URL: https://github.com/apache/shardingsphere/pull/20962
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] codecov-commenter commented on pull request #20962: Improve drop pipeline process and IT
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #20962:
URL: https://github.com/apache/shardingsphere/pull/20962#issuecomment-1246159789
# [Codecov](https://codecov.io/gh/apache/shardingsphere/pull/20962?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#20962](https://codecov.io/gh/apache/shardingsphere/pull/20962?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (26109c3) into [master](https://codecov.io/gh/apache/shardingsphere/commit/5849b5aabbb6fed04b7a3c1d4d9d8b0bbb5a615f?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5849b5a) will **decrease** coverage by `0.00%`.
> The diff coverage is `0.00%`.
```diff
@@ Coverage Diff @@
## master #20962 +/- ##
============================================
- Coverage 61.80% 61.80% -0.01%
Complexity 2464 2464
============================================
Files 4032 4032
Lines 55441 55443 +2
Branches 9398 9398
============================================
- Hits 34267 34264 -3
- Misses 18305 18310 +5
Partials 2869 2869
```
| [Impacted Files](https://codecov.io/gh/apache/shardingsphere/pull/20962?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...data/pipeline/api/PipelineJobPublicAPIFactory.java](https://codecov.io/gh/apache/shardingsphere/pull/20962/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUta2VybmVsL3NoYXJkaW5nc3BoZXJlLWRhdGEtcGlwZWxpbmUvc2hhcmRpbmdzcGhlcmUtZGF0YS1waXBlbGluZS1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NoYXJkaW5nc3BoZXJlL2RhdGEvcGlwZWxpbmUvYXBpL1BpcGVsaW5lSm9iUHVibGljQVBJRmFjdG9yeS5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...line/core/api/impl/AbstractPipelineJobAPIImpl.java](https://codecov.io/gh/apache/shardingsphere/pull/20962/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUta2VybmVsL3NoYXJkaW5nc3BoZXJlLWRhdGEtcGlwZWxpbmUvc2hhcmRpbmdzcGhlcmUtZGF0YS1waXBlbGluZS1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9zaGFyZGluZ3NwaGVyZS9kYXRhL3BpcGVsaW5lL2NvcmUvYXBpL2ltcGwvQWJzdHJhY3RQaXBlbGluZUpvYkFQSUltcGwuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...handler/distsql/ral/hint/enums/HintSourceType.java](https://codecov.io/gh/apache/shardingsphere/pull/20962/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtcHJveHkvc2hhcmRpbmdzcGhlcmUtcHJveHktYmFja2VuZC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc2hhcmRpbmdzcGhlcmUvcHJveHkvYmFja2VuZC9oYW5kbGVyL2Rpc3RzcWwvcmFsL2hpbnQvZW51bXMvSGludFNvdXJjZVR5cGUuamF2YQ==) | `0.00% <0.00%> (-42.86%)` | :arrow_down: |
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] sandynz commented on a diff in pull request #20962: Improve drop pipeline process and IT
Posted by GitBox <gi...@apache.org>.
sandynz commented on code in PR #20962:
URL: https://github.com/apache/shardingsphere/pull/20962#discussion_r970268273
##########
shardingsphere-kernel/shardingsphere-data-pipeline/shardingsphere-data-pipeline-core/src/main/java/org/apache/shardingsphere/data/pipeline/core/api/impl/PipelineProcessConfigurationPersistService.java:
##########
@@ -44,7 +45,10 @@ public PipelineProcessConfiguration load(final JobType jobType) {
@Override
public void persist(final JobType jobType, final PipelineProcessConfiguration processConfig) {
- String yamlText = YamlEngine.marshal(PROCESS_CONFIG_SWAPPER.swapToYamlConfiguration(processConfig));
+ String yamlText = "";
+ if (ObjectUtils.anyNotNull(processConfig.getRead(), processConfig.getWrite(), processConfig.getStreamChannel())) {
+ yamlText = YamlEngine.marshal(PROCESS_CONFIG_SWAPPER.swapToYamlConfiguration(processConfig));
+ }
Review Comment:
We could add a utility method `isAllFieldsNull` in `YamlPipelineProcessConfiguration` to check whether it's empty or not
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org