You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by "SteveYurongSu (via GitHub)" <gi...@apache.org> on 2024/04/02 10:26:20 UTC
[PR] Pipe: support different load balance strategies for data sync [iotdb]
SteveYurongSu opened a new pull request, #12281:
URL: https://github.com/apache/iotdb/pull/12281
## Usage
**Supported sinks:** iotdb-airgap-sink, iotdb-ssl-thrift-sink, iotdb-sync-sink, iotdb-async-sink
```
create pipe p1
with sink (
'load-balance-strategy' = 'random'
);
```
Options for `'load-balance-strategy'`: `round-robin`, `random` and `priority`. `round-robin` by default.
**Explaination:**
* `round-robin`: The connector will select the next client in a round-robin manner. If the current client is dead, the connector will try the next client until it finds an alive client. If all clients are dead, the connector will throw a `PipeConnectionException`.
* `random`: The connector will select a client randomly. If the selected client is dead, the connector will try the next client until it finds an alive client. If all clients are dead, the connector will throw a `PipeConnectionException`.
* `priority`: The connector will select the first alive client in given clients (via. `node-urls`). If all clients are dead, the connector will throw a `PipeConnectionException`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Pipe: support different load balance strategies for data sync [iotdb]
Posted by "Caideyipi (via GitHub)" <gi...@apache.org>.
Caideyipi commented on code in PR #12281:
URL: https://github.com/apache/iotdb/pull/12281#discussion_r1547609051
##########
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/pipe/config/constant/PipeConnectorConstant.java:
##########
@@ -137,6 +140,19 @@ public class PipeConnectorConstant {
public static final String SINK_LEADER_CACHE_ENABLE_KEY = "sink.leader-cache.enable";
public static final boolean CONNECTOR_LEADER_CACHE_ENABLE_DEFAULT_VALUE = true;
+ public static final String CONNECTOR_LOAD_BALANCE_STRATEGY_KEY =
+ "connector.load-balance-strategy";
+ public static final String SINK_LOAD_BALANCE_STRATEGY_KEY = "sink.load-balance-strategy";
+ public static final String CONNECTOR_LOAD_BALANCE_ROUND_ROBIN_STRATEGY = "round-robin";
+ public static final String CONNECTOR_LOAD_BALANCE_RANDOM_STRATEGY = "random";
+ public static final String CONNECTOR_LOAD_BALANCE_PRIORITY_STRATEGY = "priority";
+ public static final Set<String> CONNECTOR_LOAD_BALANCE_STRATEGY_SET =
+ new HashSet<>(
Review Comment:
unmodifiableSet
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/connector/client/IoTDBDataNodeAsyncClientManager.java:
##########
@@ -239,4 +256,55 @@ public void updateLeaderCache(String deviceId, TEndPoint endPoint) {
LEADER_CACHE_MANAGER.updateLeaderEndPoint(deviceId, endPoint);
}
+
+ /////////////////////// Strategies for load balance //////////////////////////
Review Comment:
balancing
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/connector/protocol/thrift/async/IoTDBDataRegionAsyncConnector.java:
##########
@@ -99,12 +104,27 @@ public void validate(PipeParameterValidator validator) throws Exception {
retryConnector.validate(validator);
final PipeParameters parameters = validator.getParameters();
+
validator.validate(
args -> !((boolean) args[0] || (boolean) args[1] || (boolean) args[2]),
"Only 'iotdb-thrift-ssl-sink' supports SSL transmission currently.",
parameters.getBooleanOrDefault(SINK_IOTDB_SSL_ENABLE_KEY, false),
parameters.hasAttribute(SINK_IOTDB_SSL_TRUST_STORE_PATH_KEY),
parameters.hasAttribute(SINK_IOTDB_SSL_TRUST_STORE_PWD_KEY));
+
+ loadBalanceStrategy =
+ parameters
+ .getStringOrDefault(
+ Arrays.asList(CONNECTOR_LOAD_BALANCE_STRATEGY_KEY, SINK_LOAD_BALANCE_STRATEGY_KEY),
+ CONNECTOR_LOAD_BALANCE_ROUND_ROBIN_STRATEGY)
Review Comment:
balancing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Pipe: support different load balance strategies for data sync [iotdb]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #12281:
URL: https://github.com/apache/iotdb/pull/12281#issuecomment-2031652628
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=12281) **Quality Gate failed**
Failed conditions
 [3 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_iotdb&pullRequest=12281&resolved=false&inNewCodePeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=12281&metric=new_coverage&view=list) (required ≥ 80%)
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=12281)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Pipe: support different load balance strategies for data sync [iotdb]
Posted by "SteveYurongSu (via GitHub)" <gi...@apache.org>.
SteveYurongSu merged PR #12281:
URL: https://github.com/apache/iotdb/pull/12281
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org