You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by GitBox <gi...@apache.org> on 2022/03/09 04:27:38 UTC
[GitHub] [iotdb] CRZbulabula opened a new pull request #5188: [To new_cluster] [IOTDB-2683] Hash algorithm for device group
CRZbulabula opened a new pull request #5188:
URL: https://github.com/apache/iotdb/pull/5188
Hash algorithm test result: [General hash indexes test](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=199531564)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #5188: [To new_cluster] [IOTDB-2683] Hash algorithm for device group
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #5188:
URL: https://github.com/apache/iotdb/pull/5188#issuecomment-1062571566
[](https://coveralls.io/builds/47226830)
Coverage increased (+0.1%) to 65.622% when pulling **60d21c8716da635fa60209cc30582f1d8f29ca8d on CRZbulabula:Hash_algorithm_for_device_group** into **c4e385dd71dfb62cfec53da5dc8c8095db294fdf on apache:new_cluster**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls commented on pull request #5188: [To new_cluster] [IOTDB-2683] Hash algorithm for device group
Posted by GitBox <gi...@apache.org>.
coveralls commented on pull request #5188:
URL: https://github.com/apache/iotdb/pull/5188#issuecomment-1062571566
[](https://coveralls.io/builds/47190456)
Coverage decreased (-0.02%) to 65.472% when pulling **f90233c78adb4afea5a9d17e2f14f759a1b865c0 on CRZbulabula:Hash_algorithm_for_device_group** into **c4e385dd71dfb62cfec53da5dc8c8095db294fdf on apache:new_cluster**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] wangchao316 commented on a change in pull request #5188: [To new_cluster] [IOTDB-2683] Hash algorithm for device group
Posted by GitBox <gi...@apache.org>.
wangchao316 commented on a change in pull request #5188:
URL: https://github.com/apache/iotdb/pull/5188#discussion_r823283784
##########
File path: confignode/src/assembly/resources/conf/iotdb-confignode.properties
##########
@@ -0,0 +1,38 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+####################
+### DeviceGroup Configuration
+####################
+
+# Number of DeviceGroups per StorageGroup
+# Datatype: int
+# device_group_count=10000
+
+# DeviceGroup hash algorithm
+# Datatype: String
+# Currently, BKDR, AP, JS or SDBM hash algorithm is available
+# device_group_hash_algorithm=BKDR
+
+# DeviceGroupHashExecutor package name
+# Datatype: String
+# If you want to implement your own hash algorithm, you can inherit the DeviceGroupHashExecutor class and
+# modify this parameter to correspond to your Java package prefix.
+# But make sure that your own DeviceGroupHashExecutor class name is device_group_hash_algorithm + "HashExecutor".
+# device_group_hash_executor_package=org.apache.iotdb.confignode.manager.hash.
Review comment:
hi, we could merge a config parameter. example:
device_group_hash_algorithm_class=org.apache.iotdb.confignode.manager.hash.BKDRHashExecutor.
2. remove device_group_hash_executor_package parameter
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] CRZbulabula commented on a change in pull request #5188: [To new_cluster] [IOTDB-2683] Hash algorithm for device group
Posted by GitBox <gi...@apache.org>.
CRZbulabula commented on a change in pull request #5188:
URL: https://github.com/apache/iotdb/pull/5188#discussion_r823308786
##########
File path: confignode/src/assembly/resources/conf/iotdb-confignode.properties
##########
@@ -0,0 +1,38 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+####################
+### DeviceGroup Configuration
+####################
+
+# Number of DeviceGroups per StorageGroup
+# Datatype: int
+# device_group_count=10000
+
+# DeviceGroup hash algorithm
+# Datatype: String
+# Currently, BKDR, AP, JS or SDBM hash algorithm is available
+# device_group_hash_algorithm=BKDR
+
+# DeviceGroupHashExecutor package name
+# Datatype: String
+# If you want to implement your own hash algorithm, you can inherit the DeviceGroupHashExecutor class and
+# modify this parameter to correspond to your Java package prefix.
+# But make sure that your own DeviceGroupHashExecutor class name is device_group_hash_algorithm + "HashExecutor".
+# device_group_hash_executor_package=org.apache.iotdb.confignode.manager.hash.
Review comment:
Good idea! I'm done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] OneSizeFitsQuorum merged pull request #5188: [To new_cluster] [IOTDB-2683] Hash algorithm for device group
Posted by GitBox <gi...@apache.org>.
OneSizeFitsQuorum merged pull request #5188:
URL: https://github.com/apache/iotdb/pull/5188
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #5188: [To new_cluster] [IOTDB-2683] Hash algorithm for device group
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #5188:
URL: https://github.com/apache/iotdb/pull/5188#issuecomment-1062571566
[](https://coveralls.io/builds/47190726)
Coverage increased (+0.01%) to 65.499% when pulling **f90233c78adb4afea5a9d17e2f14f759a1b865c0 on CRZbulabula:Hash_algorithm_for_device_group** into **c4e385dd71dfb62cfec53da5dc8c8095db294fdf on apache:new_cluster**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #5188: [To new_cluster] [IOTDB-2683] Hash algorithm for device group
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #5188:
URL: https://github.com/apache/iotdb/pull/5188#issuecomment-1062571566
[](https://coveralls.io/builds/47190660)
Coverage decreased (-0.03%) to 65.461% when pulling **f90233c78adb4afea5a9d17e2f14f759a1b865c0 on CRZbulabula:Hash_algorithm_for_device_group** into **c4e385dd71dfb62cfec53da5dc8c8095db294fdf on apache:new_cluster**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] wangchao316 commented on a change in pull request #5188: [To new_cluster] [IOTDB-2683] Hash algorithm for device group
Posted by GitBox <gi...@apache.org>.
wangchao316 commented on a change in pull request #5188:
URL: https://github.com/apache/iotdb/pull/5188#discussion_r822644541
##########
File path: confignode/src/main/java/org/apache/iotdb/confignode/manager/ConfigManager.java
##########
@@ -30,20 +40,48 @@
*/
public class ConfigManager {
- private final Lock partitionTableLock;
- private final PartitionTable partitionTable;
+ private static final Logger LOGGER = LoggerFactory.getLogger(ConfigManager.class);
+
+ private DeviceGroupHashExecutor hashExecutor;
+
+ private Lock partitionTableLock;
+ private PartitionTable partitionTable;
- private final LoadBalancer loadBalancer;
+ private LoadBalancer loadBalancer;
+
+ public ConfigManager(String hashType, int deviceGroupCount) {
+ setHashExecutor(hashType, deviceGroupCount);
+ }
public ConfigManager() {
+ ConfigNodeConf config = ConfigNodeDescriptor.getInstance().getConf();
+
+ setHashExecutor(config.getDeviceGroupHashAlgorithm(), config.getDeviceGroupCount());
+
this.partitionTableLock = new ReentrantLock();
this.partitionTable = new PartitionTable();
this.loadBalancer = new LoadBalancer(partitionTableLock, partitionTable);
}
+ private void setHashExecutor(String hashAlgorithm, int deviceGroupCount) {
+ switch (hashAlgorithm) {
+ case "BKDR":
+ hashExecutor = new BKDRHashExecutor(deviceGroupCount);
+ break;
+ case "AP":
+ hashExecutor = new APHashExecutor(deviceGroupCount);
+ break;
+ case "JS":
+ hashExecutor = new JSHash(deviceGroupCount);
+ break;
+ case "SDBM":
+ hashExecutor = new SDBMHash(deviceGroupCount);
Review comment:
add case default, throw a exception("does not support Hash......")
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #5188: [To new_cluster] [IOTDB-2683] Hash algorithm for device group
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #5188:
URL: https://github.com/apache/iotdb/pull/5188#issuecomment-1062571566
[](https://coveralls.io/builds/47205524)
Coverage increased (+0.03%) to 65.519% when pulling **d7ea81d1576ff9de52eaf3b79a1d5578085880ef on CRZbulabula:Hash_algorithm_for_device_group** into **c4e385dd71dfb62cfec53da5dc8c8095db294fdf on apache:new_cluster**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #5188: [To new_cluster] [IOTDB-2683] Hash algorithm for device group
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #5188:
URL: https://github.com/apache/iotdb/pull/5188#issuecomment-1062571566
[](https://coveralls.io/builds/47202007)
Coverage decreased (-0.03%) to 65.464% when pulling **262a0261139e32183fa64e63cdb78216b497472b on CRZbulabula:Hash_algorithm_for_device_group** into **c4e385dd71dfb62cfec53da5dc8c8095db294fdf on apache:new_cluster**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org