You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2020/05/05 15:09:59 UTC

[GitHub] [shardingsphere] strongduanmu opened a new pull request #5429: add standard range sharding algorithm and refactor related code

strongduanmu opened a new pull request #5429:
URL: https://github.com/apache/shardingsphere/pull/5429


   Ref #5280.
   
   Changes proposed in this pull request:
   - add standard range sharding algorithm and refactor related code


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [shardingsphere] strongduanmu commented on a change in pull request #5429: add standard range sharding algorithm and refactor related code

Posted by GitBox <gi...@apache.org>.
strongduanmu commented on a change in pull request #5429:
URL: https://github.com/apache/shardingsphere/pull/5429#discussion_r420544942



##########
File path: sharding-core/sharding-core-common/src/main/java/org/apache/shardingsphere/core/strategy/algorithm/sharding/range/CustomRangeShardingAlgorithm.java
##########
@@ -51,61 +49,39 @@
  * The sharding values will be divided into different partition by its value.
  * </p>
  */
-public final class RangeShardingAlgorithm implements StandardShardingAlgorithm<Long> {
+public final class CustomRangeShardingAlgorithm extends AbstractRangeShardingAlgorithm {
 
     private static final String PARTITION_RANGES = "partition.ranges";
 
     private Map<Integer, Range<Long>> partitionRangeMap;

Review comment:
       > Hi, do you think it is appropriate to move `partitionRangeMap` to `AbstractRangeShardingAlgorithm `?
   
   @tristaZero Before I submit the pr, I hava already tried to put the partitionRangeMap in the AbstractRangeShardingAlgorithm class, decorated with a protected identifier, but the CheckStyle detection prompt that it should be defined as a private type, and accessed through the getter/setter method. I think it is not elegant if we use the getter/setter method in the subclass to operate the partitionRangeMap.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [shardingsphere] strongduanmu commented on a change in pull request #5429: add standard range sharding algorithm and refactor related code

Posted by GitBox <gi...@apache.org>.
strongduanmu commented on a change in pull request #5429:
URL: https://github.com/apache/shardingsphere/pull/5429#discussion_r420544942



##########
File path: sharding-core/sharding-core-common/src/main/java/org/apache/shardingsphere/core/strategy/algorithm/sharding/range/CustomRangeShardingAlgorithm.java
##########
@@ -51,61 +49,39 @@
  * The sharding values will be divided into different partition by its value.
  * </p>
  */
-public final class RangeShardingAlgorithm implements StandardShardingAlgorithm<Long> {
+public final class CustomRangeShardingAlgorithm extends AbstractRangeShardingAlgorithm {
 
     private static final String PARTITION_RANGES = "partition.ranges";
 
     private Map<Integer, Range<Long>> partitionRangeMap;

Review comment:
       > Hi, do you think it is appropriate to move `partitionRangeMap` to `AbstractRangeShardingAlgorithm `?
   @tristaZero Before I submit the pr, I hava already tried to put the partitionRangeMap in the AbstractRangeShardingAlgorithm class, decorated with a protected identifier, but the CheckStyle detection prompt that it should be defined as a private type, and accessed through the getter/setter method. I think it is not elegant if we use the getter/setter method in the subclass to operate the partitionRangeMap.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [shardingsphere] tristaZero commented on a change in pull request #5429: add standard range sharding algorithm and refactor related code

Posted by GitBox <gi...@apache.org>.
tristaZero commented on a change in pull request #5429:
URL: https://github.com/apache/shardingsphere/pull/5429#discussion_r420517829



##########
File path: sharding-core/sharding-core-common/src/main/java/org/apache/shardingsphere/core/strategy/algorithm/sharding/range/CustomRangeShardingAlgorithm.java
##########
@@ -51,61 +49,39 @@
  * The sharding values will be divided into different partition by its value.
  * </p>
  */
-public final class RangeShardingAlgorithm implements StandardShardingAlgorithm<Long> {
+public final class CustomRangeShardingAlgorithm extends AbstractRangeShardingAlgorithm {
 
     private static final String PARTITION_RANGES = "partition.ranges";
 
     private Map<Integer, Range<Long>> partitionRangeMap;

Review comment:
       Hi, do you think it is appropriate to move `partitionRangeMap` to `AbstractRangeShardingAlgorithm `?

##########
File path: sharding-core/sharding-core-common/src/main/java/org/apache/shardingsphere/core/strategy/algorithm/sharding/range/StandardRangeShardingAlgorithm.java
##########
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shardingsphere.core.strategy.algorithm.sharding.range;
+
+import com.google.common.base.Preconditions;
+import com.google.common.collect.Maps;
+import com.google.common.collect.Range;
+import com.google.common.math.LongMath;
+import lombok.Getter;
+import lombok.Setter;
+import org.apache.shardingsphere.api.sharding.standard.PreciseShardingValue;
+import org.apache.shardingsphere.api.sharding.standard.RangeShardingValue;
+
+import java.math.RoundingMode;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Properties;
+
+/**
+ * Standard range sharding algorithm.
+ * <p>
+ * Standard range sharding algorithm is similar to the rule of partition table, but it can only be split by the same size.
+ * User can specify the range by setting `partition.lower`, `partition.upper` and `partition.volume` parameters.
+ * The `partition.volume` parameter determines the size of each partition.
+ * </p>
+ * <p>
+ * For example: If the `partition.lower` parameter is set to `10`, the `partition.upper` parameter is set to `45`,
+ * and the `partition.volume` parameter is set to `10`. The values in range [10,45] will be split to different partitions
+ * ——[10,20), [20, 30), [30, 40), [40, 45), and other values will be split to (-∞, 10) and [45, +∞).
+ * </p>
+ */
+public final class StandardRangeShardingAlgorithm extends AbstractRangeShardingAlgorithm {
+
+    private static final String PARTITION_LOWER = "partition.lower";
+
+    private static final String PARTITION_UPPER = "partition.upper";
+
+    private static final String PARTITION_VOLUME = "partition.volume";
+
+    private Map<Integer, Range<Long>> partitionRangeMap;
+
+    @Getter
+    @Setter
+    private Properties properties = new Properties();
+
+    @Override
+    public String doSharding(final Collection<String> availableTargetNames, final PreciseShardingValue<Long> shardingValue) {
+        checkInit();
+        return getTargetNameByPreciseShardingValue(availableTargetNames, shardingValue, partitionRangeMap);
+    }
+
+    @Override
+    public Collection<String> doSharding(final Collection<String> availableTargetNames, final RangeShardingValue<Long> shardingValue) {
+        checkInit();
+        return getTargetNameByRangeShardingValue(availableTargetNames, shardingValue, partitionRangeMap);
+    }
+
+    @Override
+    public String getType() {
+        return "STANDARD_RANGE";
+    }
+
+    @Override
+    public void initProperties() {
+        Preconditions.checkNotNull(properties.get(PARTITION_LOWER), "Standard range sharding algorithm partition lower cannot be null.");
+        Preconditions.checkNotNull(properties.get(PARTITION_UPPER), "Standard range sharding algorithm partition upper cannot be null.");
+        Preconditions.checkNotNull(properties.get(PARTITION_VOLUME), "Standard range sharding algorithm partition volume cannot be null.");
+        long lower = Long.parseLong(properties.get(PARTITION_LOWER).toString());
+        long upper = Long.parseLong(properties.get(PARTITION_UPPER).toString());
+        long volume = Long.parseLong(properties.get(PARTITION_VOLUME).toString());
+        Preconditions.checkArgument(upper - lower >= volume, "Standard range sharding algorithm partition range can not be smaller than volume.");
+

Review comment:
       A redundant blank line is expected to remove.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [shardingsphere] tristaZero commented on pull request #5429: add standard range sharding algorithm and refactor related code

Posted by GitBox <gi...@apache.org>.
tristaZero commented on pull request #5429:
URL: https://github.com/apache/shardingsphere/pull/5429#issuecomment-624415251


   @strongduanmu 
   Superb! I could pick up many highlights from your design. Just two comments need to look at. 😀 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [shardingsphere] coveralls commented on pull request #5429: add standard range sharding algorithm and refactor related code

Posted by GitBox <gi...@apache.org>.
coveralls commented on pull request #5429:
URL: https://github.com/apache/shardingsphere/pull/5429#issuecomment-624130087


   ## Pull Request Test Coverage Report for [Build 11428](https://coveralls.io/builds/30572777)
   
   * **48** of **51**   **(94.12%)**  changed or added relevant lines in **3** files are covered.
   * No unchanged relevant lines lost coverage.
   * Overall coverage increased (+**0.04%**) to **57.185%**
   
   ---
   
   |  Changes Missing Coverage | Covered Lines | Changed/Added Lines | % |
   | :-----|--------------|--------|---: |
   | [sharding-core/sharding-core-common/src/main/java/org/apache/shardingsphere/core/strategy/algorithm/sharding/range/AbstractRangeShardingAlgorithm.java](https://coveralls.io/builds/30572777/source?filename=sharding-core%2Fsharding-core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fshardingsphere%2Fcore%2Fstrategy%2Falgorithm%2Fsharding%2Frange%2FAbstractRangeShardingAlgorithm.java#L100) | 23 | 24 | 95.83%
   | [sharding-core/sharding-core-common/src/main/java/org/apache/shardingsphere/core/strategy/algorithm/sharding/range/CustomRangeShardingAlgorithm.java](https://coveralls.io/builds/30572777/source?filename=sharding-core%2Fsharding-core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fshardingsphere%2Fcore%2Fstrategy%2Falgorithm%2Fsharding%2Frange%2FCustomRangeShardingAlgorithm.java#L76) | 5 | 6 | 83.33%
   | [sharding-core/sharding-core-common/src/main/java/org/apache/shardingsphere/core/strategy/algorithm/sharding/range/StandardRangeShardingAlgorithm.java](https://coveralls.io/builds/30572777/source?filename=sharding-core%2Fsharding-core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fshardingsphere%2Fcore%2Fstrategy%2Falgorithm%2Fsharding%2Frange%2FStandardRangeShardingAlgorithm.java#L75) | 20 | 21 | 95.24%
   <!-- | **Total:** | **48** | **51** | **94.12%** | -->
   
   
   |  Totals | [![Coverage Status](https://coveralls.io/builds/30572777/badge)](https://coveralls.io/builds/30572777) |
   | :-- | --: |
   | Change from base [Build 11425](https://coveralls.io/builds/30561520): |  0.04% |
   | Covered Lines: | 11998 |
   | Relevant Lines: | 20981 |
   
   ---
   ##### 💛  - [Coveralls](https://coveralls.io)
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [shardingsphere] tristaZero commented on a change in pull request #5429: add standard range sharding algorithm and refactor related code

Posted by GitBox <gi...@apache.org>.
tristaZero commented on a change in pull request #5429:
URL: https://github.com/apache/shardingsphere/pull/5429#discussion_r420582243



##########
File path: sharding-core/sharding-core-common/src/main/java/org/apache/shardingsphere/core/strategy/algorithm/sharding/range/CustomRangeShardingAlgorithm.java
##########
@@ -51,61 +49,39 @@
  * The sharding values will be divided into different partition by its value.
  * </p>
  */
-public final class RangeShardingAlgorithm implements StandardShardingAlgorithm<Long> {
+public final class CustomRangeShardingAlgorithm extends AbstractRangeShardingAlgorithm {
 
     private static final String PARTITION_RANGES = "partition.ranges";
 
     private Map<Integer, Range<Long>> partitionRangeMap;

Review comment:
       Got your idea, it is not a big deal, and thanks for the clarification. :)




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [shardingsphere] strongduanmu commented on a change in pull request #5429: add standard range sharding algorithm and refactor related code

Posted by GitBox <gi...@apache.org>.
strongduanmu commented on a change in pull request #5429:
URL: https://github.com/apache/shardingsphere/pull/5429#discussion_r420541736



##########
File path: sharding-core/sharding-core-common/src/main/java/org/apache/shardingsphere/core/strategy/algorithm/sharding/range/StandardRangeShardingAlgorithm.java
##########
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shardingsphere.core.strategy.algorithm.sharding.range;
+
+import com.google.common.base.Preconditions;
+import com.google.common.collect.Maps;
+import com.google.common.collect.Range;
+import com.google.common.math.LongMath;
+import lombok.Getter;
+import lombok.Setter;
+import org.apache.shardingsphere.api.sharding.standard.PreciseShardingValue;
+import org.apache.shardingsphere.api.sharding.standard.RangeShardingValue;
+
+import java.math.RoundingMode;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Properties;
+
+/**
+ * Standard range sharding algorithm.
+ * <p>
+ * Standard range sharding algorithm is similar to the rule of partition table, but it can only be split by the same size.
+ * User can specify the range by setting `partition.lower`, `partition.upper` and `partition.volume` parameters.
+ * The `partition.volume` parameter determines the size of each partition.
+ * </p>
+ * <p>
+ * For example: If the `partition.lower` parameter is set to `10`, the `partition.upper` parameter is set to `45`,
+ * and the `partition.volume` parameter is set to `10`. The values in range [10,45] will be split to different partitions
+ * ——[10,20), [20, 30), [30, 40), [40, 45), and other values will be split to (-∞, 10) and [45, +∞).
+ * </p>
+ */
+public final class StandardRangeShardingAlgorithm extends AbstractRangeShardingAlgorithm {
+
+    private static final String PARTITION_LOWER = "partition.lower";
+
+    private static final String PARTITION_UPPER = "partition.upper";
+
+    private static final String PARTITION_VOLUME = "partition.volume";
+
+    private Map<Integer, Range<Long>> partitionRangeMap;
+
+    @Getter
+    @Setter
+    private Properties properties = new Properties();
+
+    @Override
+    public String doSharding(final Collection<String> availableTargetNames, final PreciseShardingValue<Long> shardingValue) {
+        checkInit();
+        return getTargetNameByPreciseShardingValue(availableTargetNames, shardingValue, partitionRangeMap);
+    }
+
+    @Override
+    public Collection<String> doSharding(final Collection<String> availableTargetNames, final RangeShardingValue<Long> shardingValue) {
+        checkInit();
+        return getTargetNameByRangeShardingValue(availableTargetNames, shardingValue, partitionRangeMap);
+    }
+
+    @Override
+    public String getType() {
+        return "STANDARD_RANGE";
+    }
+
+    @Override
+    public void initProperties() {
+        Preconditions.checkNotNull(properties.get(PARTITION_LOWER), "Standard range sharding algorithm partition lower cannot be null.");
+        Preconditions.checkNotNull(properties.get(PARTITION_UPPER), "Standard range sharding algorithm partition upper cannot be null.");
+        Preconditions.checkNotNull(properties.get(PARTITION_VOLUME), "Standard range sharding algorithm partition volume cannot be null.");
+        long lower = Long.parseLong(properties.get(PARTITION_LOWER).toString());
+        long upper = Long.parseLong(properties.get(PARTITION_UPPER).toString());
+        long volume = Long.parseLong(properties.get(PARTITION_VOLUME).toString());
+        Preconditions.checkArgument(upper - lower >= volume, "Standard range sharding algorithm partition range can not be smaller than volume.");
+

Review comment:
       > A redundant blank line is expected to remove.
   
   Ok, I will modify it.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [shardingsphere] coveralls edited a comment on pull request #5429: add standard range sharding algorithm and refactor related code

Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #5429:
URL: https://github.com/apache/shardingsphere/pull/5429#issuecomment-624130087


   ## Pull Request Test Coverage Report for [Build 11431](https://coveralls.io/builds/30588505)
   
   * **48** of **51**   **(94.12%)**  changed or added relevant lines in **3** files are covered.
   * **16** unchanged lines in **2** files lost coverage.
   * Overall coverage decreased (**-0.1%**) to **56.993%**
   
   ---
   
   |  Changes Missing Coverage | Covered Lines | Changed/Added Lines | % |
   | :-----|--------------|--------|---: |
   | [sharding-core/sharding-core-common/src/main/java/org/apache/shardingsphere/core/strategy/algorithm/sharding/range/AbstractRangeShardingAlgorithm.java](https://coveralls.io/builds/30588505/source?filename=sharding-core%2Fsharding-core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fshardingsphere%2Fcore%2Fstrategy%2Falgorithm%2Fsharding%2Frange%2FAbstractRangeShardingAlgorithm.java#L100) | 23 | 24 | 95.83%
   | [sharding-core/sharding-core-common/src/main/java/org/apache/shardingsphere/core/strategy/algorithm/sharding/range/CustomRangeShardingAlgorithm.java](https://coveralls.io/builds/30588505/source?filename=sharding-core%2Fsharding-core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fshardingsphere%2Fcore%2Fstrategy%2Falgorithm%2Fsharding%2Frange%2FCustomRangeShardingAlgorithm.java#L76) | 5 | 6 | 83.33%
   | [sharding-core/sharding-core-common/src/main/java/org/apache/shardingsphere/core/strategy/algorithm/sharding/range/StandardRangeShardingAlgorithm.java](https://coveralls.io/builds/30588505/source?filename=sharding-core%2Fsharding-core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fshardingsphere%2Fcore%2Fstrategy%2Falgorithm%2Fsharding%2Frange%2FStandardRangeShardingAlgorithm.java#L75) | 20 | 21 | 95.24%
   <!-- | **Total:** | **48** | **51** | **94.12%** | -->
   
   |  Files with Coverage Reduction | New Missed Lines | % |
   | :-----|--------------|--: |
   | [sharding-orchestration/sharding-orchestration-core/sharding-orchestration-core-configcenter/src/main/java/org/apache/shardingsphere/orchestration/core/configcenter/listener/SchemaChangedListener.java](https://coveralls.io/builds/30588505/source?filename=sharding-orchestration%2Fsharding-orchestration-core%2Fsharding-orchestration-core-configcenter%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fshardingsphere%2Forchestration%2Fcore%2Fconfigcenter%2Flistener%2FSchemaChangedListener.java#L76) | 4 | 83.93% |
   | [sharding-orchestration/sharding-orchestration-core/sharding-orchestration-core-configcenter/src/main/java/org/apache/shardingsphere/orchestration/core/configcenter/ConfigCenter.java](https://coveralls.io/builds/30588505/source?filename=sharding-orchestration%2Fsharding-orchestration-core%2Fsharding-orchestration-core-configcenter%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fshardingsphere%2Forchestration%2Fcore%2Fconfigcenter%2FConfigCenter.java#L105) | 12 | 85.37% |
   <!-- | **Total:** | **16** |  | -->
   
   |  Totals | [![Coverage Status](https://coveralls.io/builds/30588505/badge)](https://coveralls.io/builds/30588505) |
   | :-- | --: |
   | Change from base [Build 11425](https://coveralls.io/builds/30561520): |  -0.1% |
   | Covered Lines: | 11992 |
   | Relevant Lines: | 21041 |
   
   ---
   ##### 💛  - [Coveralls](https://coveralls.io)
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [shardingsphere] tristaZero commented on pull request #5429: add standard range sharding algorithm and refactor related code

Posted by GitBox <gi...@apache.org>.
tristaZero commented on pull request #5429:
URL: https://github.com/apache/shardingsphere/pull/5429#issuecomment-624480722


   Got your idea; it is not a big deal. Moreover, thanks for your much effort and deliberation for sharding algorithm. Great job!
   @strongduanmu 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org