You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2022/01/10 12:52:00 UTC

[GitHub] [shardingsphere] ReyYang opened a new pull request #14676: For #14605: The binary type is abnormal during the data migration data check phase

ReyYang opened a new pull request #14676:
URL: https://github.com/apache/shardingsphere/pull/14676


   Fixes #14605.
   
   Changes proposed in this pull request:
   - Override equals and hashCode methods of "org.apache.shardingsphere.data.pipeline.core.spi.check.consistency.DataMatchSingleTableDataCalculator.CalculatedResult"
   - Modify the use of "JDBCRowsLoader" for the return type of BINARY, VARBINARY, LONGVARBINARY
   - Modify class "JDBCMemoryQueryResult"
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [shardingsphere] sandynz merged pull request #14676: For #14605: The binary type is abnormal during the data migration data check phase

Posted by GitBox <gi...@apache.org>.
sandynz merged pull request #14676:
URL: https://github.com/apache/shardingsphere/pull/14676


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [shardingsphere] ReyYang commented on pull request #14676: For #14605: The binary type is abnormal during the data migration data check phase

Posted by GitBox <gi...@apache.org>.
ReyYang commented on pull request #14676:
URL: https://github.com/apache/shardingsphere/pull/14676#issuecomment-1009553114


   Reference documentation：https://dev.mysql.com/doc/connector-j/8.0/en/connector-j-reference-type-conversions.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [shardingsphere] sandynz commented on a change in pull request #14676: For #14605: The binary type is abnormal during the data migration data check phase

Posted by GitBox <gi...@apache.org>.
sandynz commented on a change in pull request #14676:
URL: https://github.com/apache/shardingsphere/pull/14676#discussion_r781721153



##########
File path: shardingsphere-kernel/shardingsphere-data-pipeline/shardingsphere-data-pipeline-core/src/main/java/org/apache/shardingsphere/data/pipeline/core/spi/check/consistency/DataMatchSingleTableDataCalculator.java
##########
@@ -124,5 +125,39 @@ public void init() {
         private final int recordCount;
         
         private final Collection<Collection<Object>> records;

Review comment:
       Refactor later




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [shardingsphere] codecov-commenter commented on pull request #14676: For #14605: The binary type is abnormal during the data migration data check phase

Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #14676:
URL: https://github.com/apache/shardingsphere/pull/14676#issuecomment-1009493006


   # [Codecov](https://codecov.io/gh/apache/shardingsphere/pull/14676?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#14676](https://codecov.io/gh/apache/shardingsphere/pull/14676?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (8d6bdde) into [master](https://codecov.io/gh/apache/shardingsphere/commit/cf472e4b8f8da6b0f2f7180bfddb8edd230666a3?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (cf472e4) will **increase** coverage by `0.12%`.
   > The diff coverage is `9.09%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/shardingsphere/pull/14676/graphs/tree.svg?width=650&height=150&src=pr&token=ZvlXpWa7so&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/shardingsphere/pull/14676?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@             Coverage Diff              @@
   ##             master   #14676      +/-   ##
   ============================================
   + Coverage     59.54%   59.66%   +0.12%     
   + Complexity     1849     1823      -26     
   ============================================
     Files          3032     3043      +11     
     Lines         45180    45341     +161     
     Branches       7637     7668      +31     
   ============================================
   + Hits          26901    27052     +151     
   + Misses        16178    16175       -3     
   - Partials       2101     2114      +13     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/shardingsphere/pull/14676?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...cy/AbstractStreamingSingleTableDataCalculator.java](https://codecov.io/gh/apache/shardingsphere/pull/14676/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUta2VybmVsL3NoYXJkaW5nc3BoZXJlLWRhdGEtcGlwZWxpbmUvc2hhcmRpbmdzcGhlcmUtZGF0YS1waXBlbGluZS1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9zaGFyZGluZ3NwaGVyZS9kYXRhL3BpcGVsaW5lL2NvcmUvc3BpL2NoZWNrL2NvbnNpc3RlbmN5L0Fic3RyYWN0U3RyZWFtaW5nU2luZ2xlVGFibGVEYXRhQ2FsY3VsYXRvci5qYXZh) | `0.00% <0.00%> (ø)` | |
   | [...onsistency/DataMatchSingleTableDataCalculator.java](https://codecov.io/gh/apache/shardingsphere/pull/14676/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUta2VybmVsL3NoYXJkaW5nc3BoZXJlLWRhdGEtcGlwZWxpbmUvc2hhcmRpbmdzcGhlcmUtZGF0YS1waXBlbGluZS1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9zaGFyZGluZ3NwaGVyZS9kYXRhL3BpcGVsaW5lL2NvcmUvc3BpL2NoZWNrL2NvbnNpc3RlbmN5L0RhdGFNYXRjaFNpbmdsZVRhYmxlRGF0YUNhbGN1bGF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [...y/impl/driver/jdbc/type/memory/JDBCRowsLoader.java](https://codecov.io/gh/apache/shardingsphere/pull/14676/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtaW5mcmEvc2hhcmRpbmdzcGhlcmUtaW5mcmEtZXhlY3V0b3Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NoYXJkaW5nc3BoZXJlL2luZnJhL2V4ZWN1dG9yL3NxbC9leGVjdXRlL3Jlc3VsdC9xdWVyeS9pbXBsL2RyaXZlci9qZGJjL3R5cGUvbWVtb3J5L0pEQkNSb3dzTG9hZGVyLmphdmE=) | `96.77% <100.00%> (+0.10%)` | :arrow_up: |
   | [...ce/prop/impl/DataSourcePropertiesSetterHolder.java](https://codecov.io/gh/apache/shardingsphere/pull/14676/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtamRiYy9zaGFyZGluZ3NwaGVyZS1qZGJjLXNwcmluZy9zaGFyZGluZ3NwaGVyZS1qZGJjLXNwcmluZy1pbmZyYS9zaGFyZGluZ3NwaGVyZS1qZGJjLXNwcmluZy1ib290LXN0YXJ0ZXItaW5mcmEvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NoYXJkaW5nc3BoZXJlL3NwcmluZy9ib290L2RhdGFzb3VyY2UvcHJvcC9pbXBsL0RhdGFTb3VyY2VQcm9wZXJ0aWVzU2V0dGVySG9sZGVyLmphdmE=) | `66.66% <0.00%> (-19.05%)` | :arrow_down: |
   | [...metadata/dialect/PostgreSQLDataSourceMetaData.java](https://codecov.io/gh/apache/shardingsphere/pull/14676/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtaW5mcmEvc2hhcmRpbmdzcGhlcmUtaW5mcmEtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9zaGFyZGluZ3NwaGVyZS9pbmZyYS9kYXRhYmFzZS9tZXRhZGF0YS9kaWFsZWN0L1Bvc3RncmVTUUxEYXRhU291cmNlTWV0YURhdGEuamF2YQ==) | `88.88% <0.00%> (-11.12%)` | :arrow_down: |
   | [...a/binder/statement/dml/SelectStatementContext.java](https://codecov.io/gh/apache/shardingsphere/pull/14676/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtaW5mcmEvc2hhcmRpbmdzcGhlcmUtaW5mcmEtYmluZGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9zaGFyZGluZ3NwaGVyZS9pbmZyYS9iaW5kZXIvc3RhdGVtZW50L2RtbC9TZWxlY3RTdGF0ZW1lbnRDb250ZXh0LmphdmE=) | `62.62% <0.00%> (-7.83%)` | :arrow_down: |
   | [.../metadata/dialect/SQLServerDataSourceMetaData.java](https://codecov.io/gh/apache/shardingsphere/pull/14676/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtaW5mcmEvc2hhcmRpbmdzcGhlcmUtaW5mcmEtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9zaGFyZGluZ3NwaGVyZS9pbmZyYS9kYXRhYmFzZS9tZXRhZGF0YS9kaWFsZWN0L1NRTFNlcnZlckRhdGFTb3VyY2VNZXRhRGF0YS5qYXZh) | `83.33% <0.00%> (-7.58%)` | :arrow_down: |
   | [...ng/distsql/handler/enums/ShardingStrategyType.java](https://codecov.io/gh/apache/shardingsphere/pull/14676/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtZmVhdHVyZXMvc2hhcmRpbmdzcGhlcmUtc2hhcmRpbmcvc2hhcmRpbmdzcGhlcmUtc2hhcmRpbmctZGlzdHNxbC9zaGFyZGluZ3NwaGVyZS1zaGFyZGluZy1kaXN0c3FsLWhhbmRsZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3NoYXJkaW5nc3BoZXJlL3NoYXJkaW5nL2Rpc3RzcWwvaGFuZGxlci9lbnVtcy9TaGFyZGluZ1N0cmF0ZWd5VHlwZS5qYXZh) | `68.96% <0.00%> (-7.04%)` | :arrow_down: |
   | [...re/infra/rewrite/engine/RouteSQLRewriteEngine.java](https://codecov.io/gh/apache/shardingsphere/pull/14676/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtaW5mcmEvc2hhcmRpbmdzcGhlcmUtaW5mcmEtcmV3cml0ZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc2hhcmRpbmdzcGhlcmUvaW5mcmEvcmV3cml0ZS9lbmdpbmUvUm91dGVTUUxSZXdyaXRlRW5naW5lLmphdmE=) | `87.09% <0.00%> (-6.01%)` | :arrow_down: |
   | [...ase/metadata/dialect/OracleDataSourceMetaData.java](https://codecov.io/gh/apache/shardingsphere/pull/14676/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2hhcmRpbmdzcGhlcmUtaW5mcmEvc2hhcmRpbmdzcGhlcmUtaW5mcmEtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9zaGFyZGluZ3NwaGVyZS9pbmZyYS9kYXRhYmFzZS9tZXRhZGF0YS9kaWFsZWN0L09yYWNsZURhdGFTb3VyY2VNZXRhRGF0YS5qYXZh) | `90.47% <0.00%> (-4.53%)` | :arrow_down: |
   | ... and [64 more](https://codecov.io/gh/apache/shardingsphere/pull/14676/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/shardingsphere/pull/14676?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/shardingsphere/pull/14676?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [cf472e4...8d6bdde](https://codecov.io/gh/apache/shardingsphere/pull/14676?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [shardingsphere] sandynz commented on a change in pull request #14676: For #14605: The binary type is abnormal during the data migration data check phase

Posted by GitBox <gi...@apache.org>.
sandynz commented on a change in pull request #14676:
URL: https://github.com/apache/shardingsphere/pull/14676#discussion_r781174422



##########
File path: shardingsphere-kernel/shardingsphere-data-pipeline/shardingsphere-data-pipeline-core/src/main/java/org/apache/shardingsphere/data/pipeline/core/spi/check/consistency/DataMatchSingleTableDataCalculator.java
##########
@@ -124,5 +125,39 @@ public void init() {
         private final int recordCount;
         
         private final Collection<Collection<Object>> records;

Review comment:
       Could we extract `Collection<Object>` into another class that includes special `equals` and `hashCode` impl to simplify `CalculatedResult`, and also add unit test for this new class.

##########
File path: shardingsphere-kernel/shardingsphere-data-pipeline/shardingsphere-data-pipeline-core/src/main/java/org/apache/shardingsphere/data/pipeline/core/spi/check/consistency/DataMatchSingleTableDataCalculator.java
##########
@@ -124,5 +125,39 @@ public void init() {
         private final int recordCount;
         
         private final Collection<Collection<Object>> records;
+    
+        @Override
+        public boolean equals(final Object o) {
+            if (this == o) {
+                return true;
+            }
+            if (!(o instanceof CalculatedResult)) {
+                return false;
+            }
+        
+            final CalculatedResult that = (CalculatedResult) o;
+    
+            boolean equalsFirst = new EqualsBuilder().append(getRecordCount(), that.getRecordCount()).append(getMaxUniqueKeyValue(), that.getMaxUniqueKeyValue()).isEquals();
+            if (!equalsFirst) {
+                return false;
+            }
+            Iterator<Collection<Object>> thisIterator = this.records.iterator();
+            Iterator<Collection<Object>> thatIterator = that.records.iterator();
+            while (thisIterator.hasNext() && thatIterator.hasNext()) {
+                Iterator<Object> thisNextIterator = thisIterator.next().iterator();
+                Iterator<Object> thatNextIterator = thatIterator.next().iterator();
+                while (thisNextIterator.hasNext() && thatNextIterator.hasNext()) {

Review comment:
       It's better to verify `Collection<Object>`'s size, in case of they're different.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org