You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2022/02/22 06:10:28 UTC

[GitHub] [shardingsphere] ReyYang opened a new pull request #15565: For #15541: FinishCheckJob will pass data consistency check when target table is not empty.

ReyYang opened a new pull request #15565:
URL: https://github.com/apache/shardingsphere/pull/15565


   Fixes #15541.
   
   Changes proposed in this pull request:
   - FinishedCheckJob add conditional to pass data consistency check
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [shardingsphere] LeeGuoPing commented on a change in pull request #15565: For #15541: FinishCheckJob will pass data consistency check when target table is not empty.

Posted by GitBox <gi...@apache.org>.
LeeGuoPing commented on a change in pull request #15565:
URL: https://github.com/apache/shardingsphere/pull/15565#discussion_r811633344



##########
File path: shardingsphere-kernel/shardingsphere-data-pipeline/shardingsphere-data-pipeline-core/src/main/java/org/apache/shardingsphere/data/pipeline/core/job/FinishedCheckJob.java
##########
@@ -93,6 +99,19 @@ public void execute(final ShardingContext shardingContext) {
         }
     }
     
+    private boolean checkJobStatus(final String jobId) {
+        Map<Integer, JobProgress> jobProgressMap = ruleAlteredJobAPI.getProgress(jobId);
+        final AtomicBoolean flag = new AtomicBoolean(false);

Review comment:
       I think this place can use `boolean flag = false`;




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [shardingsphere] sandynz commented on a change in pull request #15565: For #15541: FinishCheckJob will pass data consistency check when target table is not empty.

Posted by GitBox <gi...@apache.org>.
sandynz commented on a change in pull request #15565:
URL: https://github.com/apache/shardingsphere/pull/15565#discussion_r812571189



##########
File path: shardingsphere-kernel/shardingsphere-data-pipeline/shardingsphere-data-pipeline-core/src/main/java/org/apache/shardingsphere/data/pipeline/core/job/FinishedCheckJob.java
##########
@@ -51,7 +51,7 @@ public void execute(final ShardingContext shardingContext) {
                 continue;
             }
             String jobId = jobInfo.getJobId();
-            if (checkJobStatus(jobId)) {
+            if (checkAllowDataCheckJobStatus(jobId)) {
                 continue;
             }

Review comment:
       Looks `checkAllowDataCheckJobStatus` means not allow, since there's `continue;`.
   
   Use `isXxx` style could be more certain.

##########
File path: shardingsphere-kernel/shardingsphere-data-pipeline/shardingsphere-data-pipeline-core/src/main/java/org/apache/shardingsphere/data/pipeline/core/job/FinishedCheckJob.java
##########
@@ -98,14 +98,11 @@ public void execute(final ShardingContext shardingContext) {
         }
     }
     
-    private boolean checkJobStatus(final String jobId) {
+    private boolean checkAllowDataCheckJobStatus(final String jobId) {
         Map<Integer, JobProgress> jobProgressMap = ruleAlteredJobAPI.getProgress(jobId);
         boolean flag = false;
         for (JobProgress each : jobProgressMap.values()) {
-            if (each == null) {
-                continue;
-            }
-            if (!JobStatus.EXECUTE_INCREMENTAL_TASK.equals(each.getStatus())) {
+            if (each == null || !JobStatus.EXECUTE_INCREMENTAL_TASK.equals(each.getStatus())) {
                 flag = true;
                 break;

Review comment:
       `each == null` should be `null == each`.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [shardingsphere] ReyYang commented on a change in pull request #15565: For #15541: FinishCheckJob will pass data consistency check when target table is not empty.

Posted by GitBox <gi...@apache.org>.
ReyYang commented on a change in pull request #15565:
URL: https://github.com/apache/shardingsphere/pull/15565#discussion_r812515787



##########
File path: shardingsphere-kernel/shardingsphere-data-pipeline/shardingsphere-data-pipeline-core/src/main/java/org/apache/shardingsphere/data/pipeline/core/job/FinishedCheckJob.java
##########
@@ -93,6 +99,19 @@ public void execute(final ShardingContext shardingContext) {
         }
     }
     
+    private boolean checkJobStatus(final String jobId) {
+        Map<Integer, JobProgress> jobProgressMap = ruleAlteredJobAPI.getProgress(jobId);
+        final AtomicBoolean flag = new AtomicBoolean(false);
+        for (JobProgress each : jobProgressMap.values()) {
+            if (JobStatus.FINISHED.equals(each.getStatus()) || JobStatus.PREPARING_FAILURE.equals(each.getStatus())
+                    || JobStatus.EXECUTE_INVENTORY_TASK_FAILURE.equals(each.getStatus()) || JobStatus.EXECUTE_INCREMENTAL_TASK_FAILURE.equals(each.getStatus())) {
+                flag.set(true);
+                break;
+            }
+        }

Review comment:
       - The judgment here should include all jobStatus because it is possible to get an exception before incrementalTask.
   - JobProgress has added judgment NULL




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [shardingsphere] ReyYang commented on pull request #15565: For #15541: FinishCheckJob will pass data consistency check when target table is not empty.

Posted by GitBox <gi...@apache.org>.
ReyYang commented on pull request #15565:
URL: https://github.com/apache/shardingsphere/pull/15565#issuecomment-1047458913


   Preparation phase failure task would have been skipped, may affect performance


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [shardingsphere] sandynz commented on a change in pull request #15565: For #15541: FinishCheckJob will pass data consistency check when target table is not empty.

Posted by GitBox <gi...@apache.org>.
sandynz commented on a change in pull request #15565:
URL: https://github.com/apache/shardingsphere/pull/15565#discussion_r811780223



##########
File path: shardingsphere-kernel/shardingsphere-data-pipeline/shardingsphere-data-pipeline-core/src/main/java/org/apache/shardingsphere/data/pipeline/core/job/FinishedCheckJob.java
##########
@@ -93,6 +99,19 @@ public void execute(final ShardingContext shardingContext) {
         }
     }
     
+    private boolean checkJobStatus(final String jobId) {
+        Map<Integer, JobProgress> jobProgressMap = ruleAlteredJobAPI.getProgress(jobId);
+        final AtomicBoolean flag = new AtomicBoolean(false);
+        for (JobProgress each : jobProgressMap.values()) {
+            if (JobStatus.FINISHED.equals(each.getStatus()) || JobStatus.PREPARING_FAILURE.equals(each.getStatus())
+                    || JobStatus.EXECUTE_INVENTORY_TASK_FAILURE.equals(each.getStatus()) || JobStatus.EXECUTE_INCREMENTAL_TASK_FAILURE.equals(each.getStatus())) {
+                flag.set(true);
+                break;
+            }
+        }

Review comment:
       Looks it could be simplified. All sharding items' progress statuses are `EXECUTE_INCREMENTAL_TASK`, then it could do data consistency check, else not.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [shardingsphere] sandynz merged pull request #15565: For #15541: FinishCheckJob will pass data consistency check when target table is not empty.

Posted by GitBox <gi...@apache.org>.
sandynz merged pull request #15565:
URL: https://github.com/apache/shardingsphere/pull/15565


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [shardingsphere] codecov-commenter commented on pull request #15565: For #15541: FinishCheckJob will pass data consistency check when target table is not empty.

Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #15565:
URL: https://github.com/apache/shardingsphere/pull/15565#issuecomment-1047629868


   # [Codecov](https://codecov.io/gh/apache/shardingsphere/pull/15565?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#15565](https://codecov.io/gh/apache/shardingsphere/pull/15565?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2be68d7) into [master](https://codecov.io/gh/apache/shardingsphere/commit/f53f67a56f6cb3115f9b0cd2248c843fb46488b0?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f53f67a) will **decrease** coverage by `0.00%`.
   > The diff coverage is `0.00%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/shardingsphere/pull/15565/graphs/tree.svg?width=650&height=150&src=pr&token=ZvlXpWa7so&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/shardingsphere/pull/15565?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@             Coverage Diff              @@
   ##             master   #15565      +/-   ##
   ============================================
   - Coverage     60.43%   60.43%   -0.01%     
   + Complexity     1961     1960       -1     
   ============================================
     Files          3232     3231       -1     
     Lines         48574    48628      +54     
     Branches       8293     8313      +20     
   ============================================
   + Hits          29356    29387      +31     
   - Misses        16817    16835      +18     
   - Partials       2401     2406       +5     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/shardingsphere/pull/15565?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...phere/data/pipeline/core/job/FinishedCheckJob.java](https://codecov.io/gh/apache/shardingsphere/pull/15565/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUta2VybmVsL3NoYXJkaW5nc3BoZXJlLWRhdGEtcGlwZWxpbmUvc2hhcmRpbmdzcGhlcmUtZGF0YS1waXBlbGluZS1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9zaGFyZGluZ3NwaGVyZS9kYXRhL3BpcGVsaW5lL2NvcmUvam9iL0ZpbmlzaGVkQ2hlY2tKb2IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [...distsql/ral/ShowAllVariablesStatementTestCase.java](https://codecov.io/gh/apache/shardingsphere/pull/15565/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtdGVzdC9zaGFyZGluZ3NwaGVyZS1wYXJzZXItdGVzdC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc2hhcmRpbmdzcGhlcmUvdGVzdC9zcWwvcGFyc2VyL3BhcmFtZXRlcml6ZWQvamF4Yi9jYXNlcy9kb21haW4vc3RhdGVtZW50L2Rpc3RzcWwvcmFsL1Nob3dBbGxWYXJpYWJsZXNTdGF0ZW1lbnRUZXN0Q2FzZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...executor/batch/BatchPreparedStatementExecutor.java](https://codecov.io/gh/apache/shardingsphere/pull/15565/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtamRiYy9zaGFyZGluZ3NwaGVyZS1qZGJjLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NoYXJkaW5nc3BoZXJlL2RyaXZlci9leGVjdXRvci9iYXRjaC9CYXRjaFByZXBhcmVkU3RhdGVtZW50RXhlY3V0b3IuamF2YQ==) | `94.23% <0.00%> (-3.60%)` | :arrow_down: |
   | [.../manager/cluster/ClusterContextManagerBuilder.java](https://codecov.io/gh/apache/shardingsphere/pull/15565/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtbW9kZS9zaGFyZGluZ3NwaGVyZS1tb2RlLXR5cGUvc2hhcmRpbmdzcGhlcmUtY2x1c3Rlci1tb2RlL3NoYXJkaW5nc3BoZXJlLWNsdXN0ZXItbW9kZS1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9zaGFyZGluZ3NwaGVyZS9tb2RlL21hbmFnZXIvY2x1c3Rlci9DbHVzdGVyQ29udGV4dE1hbmFnZXJCdWlsZGVyLmphdmE=) | `55.07% <0.00%> (-2.51%)` | :arrow_down: |
   | [...end/text/distsql/ral/RALBackendHandlerFactory.java](https://codecov.io/gh/apache/shardingsphere/pull/15565/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtcHJveHkvc2hhcmRpbmdzcGhlcmUtcHJveHktYmFja2VuZC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc2hhcmRpbmdzcGhlcmUvcHJveHkvYmFja2VuZC90ZXh0L2Rpc3RzcWwvcmFsL1JBTEJhY2tlbmRIYW5kbGVyRmFjdG9yeS5qYXZh) | `91.66% <0.00%> (-0.18%)` | :arrow_down: |
   | [...ore/statement/ShardingSpherePreparedStatement.java](https://codecov.io/gh/apache/shardingsphere/pull/15565/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtamRiYy9zaGFyZGluZ3NwaGVyZS1qZGJjLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NoYXJkaW5nc3BoZXJlL2RyaXZlci9qZGJjL2NvcmUvc3RhdGVtZW50L1NoYXJkaW5nU3BoZXJlUHJlcGFyZWRTdGF0ZW1lbnQuamF2YQ==) | `66.28% <0.00%> (-0.13%)` | :arrow_down: |
   | [...eterized/jaxb/cases/domain/SQLParserTestCases.java](https://codecov.io/gh/apache/shardingsphere/pull/15565/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtdGVzdC9zaGFyZGluZ3NwaGVyZS1wYXJzZXItdGVzdC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc2hhcmRpbmdzcGhlcmUvdGVzdC9zcWwvcGFyc2VyL3BhcmFtZXRlcml6ZWQvamF4Yi9jYXNlcy9kb21haW4vU1FMUGFyc2VyVGVzdENhc2VzLmphdmE=) | `99.82% <0.00%> (-0.01%)` | :arrow_down: |
   | [...ser/core/common/CommonDistSQLStatementVisitor.java](https://codecov.io/gh/apache/shardingsphere/pull/15565/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtZGlzdHNxbC9zaGFyZGluZ3NwaGVyZS1kaXN0c3FsLXBhcnNlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc2hhcmRpbmdzcGhlcmUvZGlzdHNxbC9wYXJzZXIvY29yZS9jb21tb24vQ29tbW9uRGlzdFNRTFN0YXRlbWVudFZpc2l0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [...ent/distsql/ral/ShowVariableStatementTestCase.java](https://codecov.io/gh/apache/shardingsphere/pull/15565/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtdGVzdC9zaGFyZGluZ3NwaGVyZS1wYXJzZXItdGVzdC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc2hhcmRpbmdzcGhlcmUvdGVzdC9zcWwvcGFyc2VyL3BhcmFtZXRlcml6ZWQvamF4Yi9jYXNlcy9kb21haW4vc3RhdGVtZW50L2Rpc3RzcWwvcmFsL1Nob3dWYXJpYWJsZVN0YXRlbWVudFRlc3RDYXNlLmphdmE=) | `100.00% <0.00%> (ø)` | |
   | [...y/workerid/generator/ClusterWorkerIdGenerator.java](https://codecov.io/gh/apache/shardingsphere/pull/15565/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtbW9kZS9zaGFyZGluZ3NwaGVyZS1tb2RlLXR5cGUvc2hhcmRpbmdzcGhlcmUtY2x1c3Rlci1tb2RlL3NoYXJkaW5nc3BoZXJlLWNsdXN0ZXItbW9kZS1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9zaGFyZGluZ3NwaGVyZS9tb2RlL21hbmFnZXIvY2x1c3Rlci9jb29yZGluYXRvci9yZWdpc3RyeS93b3JrZXJpZC9nZW5lcmF0b3IvQ2x1c3RlcldvcmtlcklkR2VuZXJhdG9yLmphdmE=) | `0.00% <0.00%> (ø)` | |
   | ... and [74 more](https://codecov.io/gh/apache/shardingsphere/pull/15565/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/shardingsphere/pull/15565?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/shardingsphere/pull/15565?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [f53f67a...2be68d7](https://codecov.io/gh/apache/shardingsphere/pull/15565?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [shardingsphere] sandynz commented on a change in pull request #15565: For #15541: FinishCheckJob will pass data consistency check when target table is not empty.

Posted by GitBox <gi...@apache.org>.
sandynz commented on a change in pull request #15565:
URL: https://github.com/apache/shardingsphere/pull/15565#discussion_r811785106



##########
File path: shardingsphere-kernel/shardingsphere-data-pipeline/shardingsphere-data-pipeline-core/src/main/java/org/apache/shardingsphere/data/pipeline/core/job/FinishedCheckJob.java
##########
@@ -93,6 +99,19 @@ public void execute(final ShardingContext shardingContext) {
         }
     }
     
+    private boolean checkJobStatus(final String jobId) {
+        Map<Integer, JobProgress> jobProgressMap = ruleAlteredJobAPI.getProgress(jobId);
+        final AtomicBoolean flag = new AtomicBoolean(false);
+        for (JobProgress each : jobProgressMap.values()) {
+            if (JobStatus.FINISHED.equals(each.getStatus()) || JobStatus.PREPARING_FAILURE.equals(each.getStatus())
+                    || JobStatus.EXECUTE_INVENTORY_TASK_FAILURE.equals(each.getStatus()) || JobStatus.EXECUTE_INCREMENTAL_TASK_FAILURE.equals(each.getStatus())) {
+                flag.set(true);
+                break;
+            }
+        }

Review comment:
       And `JobProgress each` might be null.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [shardingsphere] sandynz commented on a change in pull request #15565: For #15541: FinishCheckJob will pass data consistency check when target table is not empty.

Posted by GitBox <gi...@apache.org>.
sandynz commented on a change in pull request #15565:
URL: https://github.com/apache/shardingsphere/pull/15565#discussion_r812530265



##########
File path: shardingsphere-kernel/shardingsphere-data-pipeline/shardingsphere-data-pipeline-core/src/main/java/org/apache/shardingsphere/data/pipeline/core/job/FinishedCheckJob.java
##########
@@ -101,15 +100,19 @@ public void execute(final ShardingContext shardingContext) {
     
     private boolean checkJobStatus(final String jobId) {
         Map<Integer, JobProgress> jobProgressMap = ruleAlteredJobAPI.getProgress(jobId);
-        final AtomicBoolean flag = new AtomicBoolean(false);
+        boolean flag = false;
         for (JobProgress each : jobProgressMap.values()) {
+            if (each == null) {
+                continue;
+            }
             if (JobStatus.FINISHED.equals(each.getStatus()) || JobStatus.PREPARING_FAILURE.equals(each.getStatus())
-                    || JobStatus.EXECUTE_INVENTORY_TASK_FAILURE.equals(each.getStatus()) || JobStatus.EXECUTE_INCREMENTAL_TASK_FAILURE.equals(each.getStatus())) {
-                flag.set(true);
+                    || JobStatus.EXECUTE_INVENTORY_TASK_FAILURE.equals(each.getStatus())
+                    || JobStatus.EXECUTE_INCREMENTAL_TASK_FAILURE.equals(each.getStatus())) {
+                flag = true;
                 break;
             }

Review comment:
       It's better to depend on less status as previous comment discussed. Not only simplify code, but also if there's new status added, then it's not necessary to change code here.
   
   `jobProgressMap` for each could be used to reduce code lines.
   




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [shardingsphere] codecov-commenter edited a comment on pull request #15565: For #15541: FinishCheckJob will pass data consistency check when target table is not empty.

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #15565:
URL: https://github.com/apache/shardingsphere/pull/15565#issuecomment-1047629868


   # [Codecov](https://codecov.io/gh/apache/shardingsphere/pull/15565?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#15565](https://codecov.io/gh/apache/shardingsphere/pull/15565?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (76bbcbd) into [master](https://codecov.io/gh/apache/shardingsphere/commit/f53f67a56f6cb3115f9b0cd2248c843fb46488b0?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f53f67a) will **decrease** coverage by `0.02%`.
   > The diff coverage is `0.00%`.
   
   > :exclamation: Current head 76bbcbd differs from pull request most recent head deec97f. Consider uploading reports for the commit deec97f to get more accurate results
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/shardingsphere/pull/15565/graphs/tree.svg?width=650&height=150&src=pr&token=ZvlXpWa7so&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/shardingsphere/pull/15565?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@             Coverage Diff              @@
   ##             master   #15565      +/-   ##
   ============================================
   - Coverage     60.43%   60.40%   -0.03%     
   - Complexity     1961     1986      +25     
   ============================================
     Files          3232     3232              
     Lines         48574    48673      +99     
     Branches       8293     8320      +27     
   ============================================
   + Hits          29356    29403      +47     
   - Misses        16817    16856      +39     
   - Partials       2401     2414      +13     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/shardingsphere/pull/15565?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...phere/data/pipeline/core/job/FinishedCheckJob.java](https://codecov.io/gh/apache/shardingsphere/pull/15565/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUta2VybmVsL3NoYXJkaW5nc3BoZXJlLWRhdGEtcGlwZWxpbmUvc2hhcmRpbmdzcGhlcmUtZGF0YS1waXBlbGluZS1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9zaGFyZGluZ3NwaGVyZS9kYXRhL3BpcGVsaW5lL2NvcmUvam9iL0ZpbmlzaGVkQ2hlY2tKb2IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [...distsql/ral/ShowAllVariablesStatementTestCase.java](https://codecov.io/gh/apache/shardingsphere/pull/15565/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtdGVzdC9zaGFyZGluZ3NwaGVyZS1wYXJzZXItdGVzdC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc2hhcmRpbmdzcGhlcmUvdGVzdC9zcWwvcGFyc2VyL3BhcmFtZXRlcml6ZWQvamF4Yi9jYXNlcy9kb21haW4vc3RhdGVtZW50L2Rpc3RzcWwvcmFsL1Nob3dBbGxWYXJpYWJsZXNTdGF0ZW1lbnRUZXN0Q2FzZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...d/text/distsql/ral/common/hint/HintSourceType.java](https://codecov.io/gh/apache/shardingsphere/pull/15565/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtcHJveHkvc2hhcmRpbmdzcGhlcmUtcHJveHktYmFja2VuZC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc2hhcmRpbmdzcGhlcmUvcHJveHkvYmFja2VuZC90ZXh0L2Rpc3RzcWwvcmFsL2NvbW1vbi9oaW50L0hpbnRTb3VyY2VUeXBlLmphdmE=) | `0.00% <0.00%> (-42.86%)` | :arrow_down: |
   | [...g/algorithm/sharding/mod/ModShardingAlgorithm.java](https://codecov.io/gh/apache/shardingsphere/pull/15565/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtZmVhdHVyZXMvc2hhcmRpbmdzcGhlcmUtc2hhcmRpbmcvc2hhcmRpbmdzcGhlcmUtc2hhcmRpbmctY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc2hhcmRpbmdzcGhlcmUvc2hhcmRpbmcvYWxnb3JpdGhtL3NoYXJkaW5nL21vZC9Nb2RTaGFyZGluZ0FsZ29yaXRobS5qYXZh) | `76.92% <0.00%> (-4.56%)` | :arrow_down: |
   | [...tatement/impl/SQLServerTCLStatementSQLVisitor.java](https://codecov.io/gh/apache/shardingsphere/pull/15565/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtc3FsLXBhcnNlci9zaGFyZGluZ3NwaGVyZS1zcWwtcGFyc2VyLWRpYWxlY3Qvc2hhcmRpbmdzcGhlcmUtc3FsLXBhcnNlci1zcWxzZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NoYXJkaW5nc3BoZXJlL3NxbC9wYXJzZXIvc3Fsc2VydmVyL3Zpc2l0b3Ivc3RhdGVtZW50L2ltcGwvU1FMU2VydmVyVENMU3RhdGVtZW50U1FMVmlzaXRvci5qYXZh) | `88.46% <0.00%> (-4.40%)` | :arrow_down: |
   | [...apache/shardingsphere/sharding/rule/TableRule.java](https://codecov.io/gh/apache/shardingsphere/pull/15565/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtZmVhdHVyZXMvc2hhcmRpbmdzcGhlcmUtc2hhcmRpbmcvc2hhcmRpbmdzcGhlcmUtc2hhcmRpbmctY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc2hhcmRpbmdzcGhlcmUvc2hhcmRpbmcvcnVsZS9UYWJsZVJ1bGUuamF2YQ==) | `83.00% <0.00%> (-4.36%)` | :arrow_down: |
   | [...executor/batch/BatchPreparedStatementExecutor.java](https://codecov.io/gh/apache/shardingsphere/pull/15565/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtamRiYy9zaGFyZGluZ3NwaGVyZS1qZGJjLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NoYXJkaW5nc3BoZXJlL2RyaXZlci9leGVjdXRvci9iYXRjaC9CYXRjaFByZXBhcmVkU3RhdGVtZW50RXhlY3V0b3IuamF2YQ==) | `94.23% <0.00%> (-3.60%)` | :arrow_down: |
   | [.../manager/cluster/ClusterContextManagerBuilder.java](https://codecov.io/gh/apache/shardingsphere/pull/15565/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtbW9kZS9zaGFyZGluZ3NwaGVyZS1tb2RlLXR5cGUvc2hhcmRpbmdzcGhlcmUtY2x1c3Rlci1tb2RlL3NoYXJkaW5nc3BoZXJlLWNsdXN0ZXItbW9kZS1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9zaGFyZGluZ3NwaGVyZS9tb2RlL21hbmFnZXIvY2x1c3Rlci9DbHVzdGVyQ29udGV4dE1hbmFnZXJCdWlsZGVyLmphdmE=) | `55.07% <0.00%> (-2.51%)` | :arrow_down: |
   | [...ore/statement/ShardingSpherePreparedStatement.java](https://codecov.io/gh/apache/shardingsphere/pull/15565/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtamRiYy9zaGFyZGluZ3NwaGVyZS1qZGJjLWNvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NoYXJkaW5nc3BoZXJlL2RyaXZlci9qZGJjL2NvcmUvc3RhdGVtZW50L1NoYXJkaW5nU3BoZXJlUHJlcGFyZWRTdGF0ZW1lbnQuamF2YQ==) | `66.28% <0.00%> (-0.13%)` | :arrow_down: |
   | [...eterized/jaxb/cases/domain/SQLParserTestCases.java](https://codecov.io/gh/apache/shardingsphere/pull/15565/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtdGVzdC9zaGFyZGluZ3NwaGVyZS1wYXJzZXItdGVzdC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc2hhcmRpbmdzcGhlcmUvdGVzdC9zcWwvcGFyc2VyL3BhcmFtZXRlcml6ZWQvamF4Yi9jYXNlcy9kb21haW4vU1FMUGFyc2VyVGVzdENhc2VzLmphdmE=) | `99.82% <0.00%> (-0.01%)` | :arrow_down: |
   | ... and [85 more](https://codecov.io/gh/apache/shardingsphere/pull/15565/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/shardingsphere/pull/15565?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/shardingsphere/pull/15565?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [f53f67a...deec97f](https://codecov.io/gh/apache/shardingsphere/pull/15565?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [shardingsphere] sandynz commented on a change in pull request #15565: For #15541: FinishCheckJob will pass data consistency check when target table is not empty.

Posted by GitBox <gi...@apache.org>.
sandynz commented on a change in pull request #15565:
URL: https://github.com/apache/shardingsphere/pull/15565#discussion_r812543198



##########
File path: shardingsphere-kernel/shardingsphere-data-pipeline/shardingsphere-data-pipeline-core/src/main/java/org/apache/shardingsphere/data/pipeline/core/job/FinishedCheckJob.java
##########
@@ -49,6 +51,9 @@ public void execute(final ShardingContext shardingContext) {
                 continue;
             }
             String jobId = jobInfo.getJobId();
+            if (checkJobStatus(jobId)) {
+                continue;
+            }

Review comment:
       The method name `checkJobStatus` could be more meaningful.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org