You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by pa...@apache.org on 2022/06/22 06:44:46 UTC
[shardingsphere] branch master updated: Add query route primary data source when replica all offline. (#18473)
This is an automated email from the ASF dual-hosted git repository.
panjuan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/shardingsphere.git
The following commit(s) were added to refs/heads/master by this push:
new 1edfca136e4 Add query route primary data source when replica all offline. (#18473)
1edfca136e4 is described below
commit 1edfca136e4d10017355844116075db5d8beade5
Author: zhaojinchao <zh...@apache.org>
AuthorDate: Wed Jun 22 14:44:34 2022 +0800
Add query route primary data source when replica all offline. (#18473)
* Add query route primary data source when replica all offline.
* Adjust unit test
* Modify document
* Fix bug
* Adjust key name
* Adjust method name
---
.../java-api/rules/readwrite-splitting.cn.md | 12 +++++------
.../java-api/rules/readwrite-splitting.en.md | 12 +++++------
.../rules/readwrite-splitting.cn.md | 1 +
.../rules/readwrite-splitting.en.md | 1 +
.../rules/readwrite-splitting.cn.md | 2 +-
.../rules/readwrite-splitting.en.md | 2 +-
.../yaml-config/rules/readwrite-splitting.cn.md | 1 +
.../yaml-config/rules/readwrite-splitting.en.md | 1 +
.../impl/ReadwriteSplittingDataSourceRouter.java | 8 ++++++-
.../ReadwriteSplittingStrategyFactory.java | 3 ++-
.../type/DynamicReadwriteSplittingStrategy.java | 2 ++
...teSplittingDataSourceRuleConfigurationTest.java | 25 +++++++++++++++-------
12 files changed, 46 insertions(+), 24 deletions(-)
diff --git a/docs/document/content/user-manual/shardingsphere-jdbc/java-api/rules/readwrite-splitting.cn.md b/docs/document/content/user-manual/shardingsphere-jdbc/java-api/rules/readwrite-splitting.cn.md
index 7eb326cfb42..f76f065e868 100644
--- a/docs/document/content/user-manual/shardingsphere-jdbc/java-api/rules/readwrite-splitting.cn.md
+++ b/docs/document/content/user-manual/shardingsphere-jdbc/java-api/rules/readwrite-splitting.cn.md
@@ -20,12 +20,12 @@ weight = 2
可配置属性：
-| *名称* | *数据类型* | *说明* | *默认值* |
-| -------------------------- | -------------------- | ----------------------------------------------------------------------------------------------------------| ---------------|
-| name | String | 读写分离数据源名称 | - |
-| type | String | 读写分离类型，分为静态和动态。如 Static、Dynamic | - |
-| props | Properties | 读写分离所需属性，如静态：write-data-source-name、read-data-source-names，动态：auto-aware-data-source-name | - |
-| loadBalancerName (?) | String | 读库负载均衡算法名称 | 轮询负载均衡算法 |
+| *名称* | *数据类型* | *说明* | *默认值* |
+| -------------------------- | -------------------- | --------------------------------------------------------------------------------------------------------------------------------------| ---------------|
+| name | String | 读写分离数据源名称 | - |
+| type | String | 读写分离类型，分为静态和动态。如 Static、Dynamic | - |
+| props | Properties | 读写分离所需属性，如静态：write-data-source-name、read-data-source-names，动态：auto-aware-data-source-name、write-data-source-query-enabled | - |
+| loadBalancerName (?) | String | 读库负载均衡算法名称 | 轮询负载均衡算法 |
算法类型的详情，请参见[内置负载均衡算法列表](/cn/user-manual/shardingsphere-jdbc/builtin-algorithm/load-balance)。
查询一致性路由的详情，请参见[使用规范](/cn/features/readwrite-splitting/use-norms)。
diff --git a/docs/document/content/user-manual/shardingsphere-jdbc/java-api/rules/readwrite-splitting.en.md b/docs/document/content/user-manual/shardingsphere-jdbc/java-api/rules/readwrite-splitting.en.md
index 1988dda82a3..335c6582918 100644
--- a/docs/document/content/user-manual/shardingsphere-jdbc/java-api/rules/readwrite-splitting.en.md
+++ b/docs/document/content/user-manual/shardingsphere-jdbc/java-api/rules/readwrite-splitting.en.md
@@ -20,12 +20,12 @@ Class name: org.apache.shardingsphere.readwritesplitting.api.rule.ReadwriteSplit
Attributes:
-| *Name* | *DataType* | *Description* | *Default Value* |
-| -------------------------- | -------------------- | -------------------------------------------------- | ---------------------------------- |
-| name | String | Readwrite-splitting data source name | - |
-| type | String | Readwrite-splitting type, such as: Static, Dynamic | - |
-| props | Properties | Readwrite-splitting required properties. Static: write-data-source-name, read-data-source-names, Dynamic: auto-aware-data-source-name| - |
-| loadBalancerName (?) | String | Load balance algorithm name of replica sources | Round robin load balance algorithm |
+| *Name* | *DataType* | *Description* | *Default Value* |
+| -------------------------- | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------|
+| name | String | Readwrite-splitting data source name | - |
+| type | String | Readwrite-splitting type, such as: Static, Dynamic | - |
+| props | Properties | Readwrite-splitting required properties. Static: write-data-source-name, read-data-source-names, Dynamic: auto-aware-data-source-name, write-data-source-query-enabled| - |
+| loadBalancerName (?) | String | Load balance algorithm name of replica sources | Round robin load balance algorithm |
Please refer to [Built-in Load Balance Algorithm List](/en/user-manual/shardingsphere-jdbc/builtin-algorithm/load-balance) for more details about type of algorithm.
Please refer to [Use Norms](/en/features/readwrite-splitting/use-norms) for more details about query consistent routing.
diff --git a/docs/document/content/user-manual/shardingsphere-jdbc/spring-boot-starter/rules/readwrite-splitting.cn.md b/docs/document/content/user-manual/shardingsphere-jdbc/spring-boot-starter/rules/readwrite-splitting.cn.md
index c63e2b7783a..9a4ac772788 100644
--- a/docs/document/content/user-manual/shardingsphere-jdbc/spring-boot-starter/rules/readwrite-splitting.cn.md
+++ b/docs/document/content/user-manual/shardingsphere-jdbc/spring-boot-starter/rules/readwrite-splitting.cn.md
@@ -10,6 +10,7 @@ spring.shardingsphere.datasource.names= # 省略数据源配置，请参考使
spring.shardingsphere.rules.readwrite-splitting.data-sources.<readwrite-splitting-data-source-name>.type= # 读写分离类型，如: Static，Dynamic
spring.shardingsphere.rules.readwrite-splitting.data-sources.<readwrite-splitting-data-source-name>.props.auto-aware-data-source-name= # 自动发现数据源名称（与数据库发现配合使用）
+spring.shardingsphere.rules.readwrite-splitting.data-sources.<readwrite-splitting-data-source-name>.props.write-data-source-query-enabled= # 从库全部宕机、主库是否承担读流量（与数据库发现配合使用）
spring.shardingsphere.rules.readwrite-splitting.data-sources.<readwrite-splitting-data-source-name>.props.write-data-source-name= # 写数据源名称
spring.shardingsphere.rules.readwrite-splitting.data-sources.<readwrite-splitting-data-source-name>.props.read-data-source-names= # 读数据源名称，多个从数据源用逗号分隔
spring.shardingsphere.rules.readwrite-splitting.data-sources.<readwrite-splitting-data-source-name>.load-balancer-name= # 负载均衡算法名称
diff --git a/docs/document/content/user-manual/shardingsphere-jdbc/spring-boot-starter/rules/readwrite-splitting.en.md b/docs/document/content/user-manual/shardingsphere-jdbc/spring-boot-starter/rules/readwrite-splitting.en.md
index 5e1e52f7111..f5d89eb25df 100644
--- a/docs/document/content/user-manual/shardingsphere-jdbc/spring-boot-starter/rules/readwrite-splitting.en.md
+++ b/docs/document/content/user-manual/shardingsphere-jdbc/spring-boot-starter/rules/readwrite-splitting.en.md
@@ -10,6 +10,7 @@ spring.shardingsphere.datasource.names= # Omit the data source configuration, pl
spring.shardingsphere.rules.readwrite-splitting.data-sources.<readwrite-splitting-data-source-name>.type= # Readwrite-splitting type, such as: Static, Dynamic
spring.shardingsphere.rules.readwrite-splitting.data-sources.<readwrite-splitting-data-source-name>.props.auto-aware-data-source-name= # Auto aware data source name(Use with database discovery)
+spring.shardingsphere.rules.readwrite-splitting.data-sources.<readwrite-splitting-data-source-name>.props.write-data-source-query-enabled= # replicas all offline, primary data source whether provide query(Use with database discovery)
spring.shardingsphere.rules.readwrite-splitting.data-sources.<readwrite-splitting-data-source-name>.props.write-data-source-name= # Write data source name
spring.shardingsphere.rules.readwrite-splitting.data-sources.<readwrite-splitting-data-source-name>.props.read-data-source-names= # Read data source names, multiple data source names separated with comma
spring.shardingsphere.rules.readwrite-splitting.data-sources.<readwrite-splitting-data-source-name>.load-balancer-name= # Load balance algorithm name
diff --git a/docs/document/content/user-manual/shardingsphere-jdbc/spring-namespace/rules/readwrite-splitting.cn.md b/docs/document/content/user-manual/shardingsphere-jdbc/spring-namespace/rules/readwrite-splitting.cn.md
index 01e202ed96b..63c6ac5fc3c 100644
--- a/docs/document/content/user-manual/shardingsphere-jdbc/spring-namespace/rules/readwrite-splitting.cn.md
+++ b/docs/document/content/user-manual/shardingsphere-jdbc/spring-namespace/rules/readwrite-splitting.cn.md
@@ -20,7 +20,7 @@ weight = 2
| -------------------------- | ----- | -------------------------------------------- |
| id | 属性 | 读写分离数据源规则名称 |
| type | 属性 | 读写分离类型，分为静态和动态。如 Static、Dynamic |
-| props | 标签 | 读写分离所需属性，如静态：write-data-source-name、read-data-source-names，动态：auto-aware-data-source-name |
+| props | 标签 | 读写分离所需属性，如静态：write-data-source-name、read-data-source-names，动态：auto-aware-data-source-name、write-data-source-query-enabled |
| load-balance-algorithm-ref | 属性 | 负载均衡算法名称 |
diff --git a/docs/document/content/user-manual/shardingsphere-jdbc/spring-namespace/rules/readwrite-splitting.en.md b/docs/document/content/user-manual/shardingsphere-jdbc/spring-namespace/rules/readwrite-splitting.en.md
index 532e6362291..9c8447b5a37 100644
--- a/docs/document/content/user-manual/shardingsphere-jdbc/spring-namespace/rules/readwrite-splitting.en.md
+++ b/docs/document/content/user-manual/shardingsphere-jdbc/spring-namespace/rules/readwrite-splitting.en.md
@@ -20,7 +20,7 @@ Namespace: [http://shardingsphere.apache.org/schema/shardingsphere/readwrite-spl
| -------------------------- | ---------- | ----------------------------------------------------------------------- |
| id | Attribute | Readwrite-splitting data source rule name |
| type | Attribute | Readwrite-splitting type, such as: Static, Dynamic |
-| props | Tag | Readwrite-splitting required properties. Static: write-data-source-name, read-data-source-names, Dynamic: auto-aware-data-source-name |
+| props | Tag | Readwrite-splitting required properties. Static: write-data-source-name, read-data-source-names, Dynamic: auto-aware-data-source-name, write-data-source-query-enabled |
| load-balance-algorithm-ref | Attribute | Load balance algorithm name |
\<readwrite-splitting:load-balance-algorithm />
diff --git a/docs/document/content/user-manual/shardingsphere-jdbc/yaml-config/rules/readwrite-splitting.cn.md b/docs/document/content/user-manual/shardingsphere-jdbc/yaml-config/rules/readwrite-splitting.cn.md
index 1d7a71f9042..e09ed484886 100644
--- a/docs/document/content/user-manual/shardingsphere-jdbc/yaml-config/rules/readwrite-splitting.cn.md
+++ b/docs/document/content/user-manual/shardingsphere-jdbc/yaml-config/rules/readwrite-splitting.cn.md
@@ -13,6 +13,7 @@ rules:
type: # 读写分离类型，比如：Static，Dynamic
props:
auto-aware-data-source-name: # 自动发现数据源名称（与数据库发现配合使用）
+ write-data-source-query-enabled: # 从库全部宕机、主库是否承担读流量（与数据库发现配合使用）
write-data-source-name: # 写库数据源名称
read-data-source-names: # 读库数据源名称，多个从数据源用逗号分隔
loadBalancerName: # 负载均衡算法名称
diff --git a/docs/document/content/user-manual/shardingsphere-jdbc/yaml-config/rules/readwrite-splitting.en.md b/docs/document/content/user-manual/shardingsphere-jdbc/yaml-config/rules/readwrite-splitting.en.md
index 836cc3c68ad..b1669edc005 100644
--- a/docs/document/content/user-manual/shardingsphere-jdbc/yaml-config/rules/readwrite-splitting.en.md
+++ b/docs/document/content/user-manual/shardingsphere-jdbc/yaml-config/rules/readwrite-splitting.en.md
@@ -13,6 +13,7 @@ rules:
type: # Readwrite-splitting type, such as: Static, Dynamic
props:
auto-aware-data-source-name: # Auto aware data source name(Use with database discovery)
+ write-data-source-query-enabled: # replicas all offline, primary data source whether provide query(Use with database discovery)
write-data-source-name: # Write data source name
read-data-source-names: # Read data source names, multiple data source names separated with comma
loadBalancerName: # Load balance algorithm name
diff --git a/shardingsphere-features/shardingsphere-readwrite-splitting/shardingsphere-readwrite-splitting-core/src/main/java/org/apache/shardingsphere/readwritesplitting/route/impl/ReadwriteSplittingDataSourceRouter.java b/shardingsphere-features/shardingsphere-readwrite-splitting/shardingsphere-readwrite-splitting-core/src/main/java/org/apache/shardingsphere/readwritesplitting/route/impl/ReadwriteSplittingDataSourceRouter.java
index ae0a78296d3..5773ff964e1 100644
--- a/shardingsphere-features/shardingsphere-readwrite-splitting/shardingsphere-readwrite-splitting-core/src/main/java/org/apache/shardingsphere/readwritesplitting/route/impl/ReadwriteSplittingDataSourceRouter.java
+++ b/shardingsphere-features/shardingsphere-readwrite-splitting/shardingsphere-readwrite-splitting-core/src/main/java/org/apache/shardingsphere/readwritesplitting/route/impl/ReadwriteSplittingDataSourceRouter.java
@@ -23,6 +23,7 @@ import org.apache.shardingsphere.infra.binder.statement.SQLStatementContext;
import org.apache.shardingsphere.infra.binder.statement.dml.SelectStatementContext;
import org.apache.shardingsphere.infra.hint.HintManager;
import org.apache.shardingsphere.readwritesplitting.rule.ReadwriteSplittingDataSourceRule;
+import org.apache.shardingsphere.readwritesplitting.strategy.type.DynamicReadwriteSplittingStrategy;
import org.apache.shardingsphere.sql.parser.sql.common.statement.SQLStatement;
import org.apache.shardingsphere.sql.parser.sql.common.statement.dml.SelectStatement;
import org.apache.shardingsphere.sql.parser.sql.dialect.handler.dml.SelectStatementHandler;
@@ -49,7 +50,7 @@ public final class ReadwriteSplittingDataSourceRouter {
}
private boolean isPrimaryRoute(final SQLStatementContext<?> sqlStatementContext) {
- return isWriteRouteStatement(sqlStatementContext) || isHintWriteRouteOnly(sqlStatementContext);
+ return isWriteRouteStatement(sqlStatementContext) || isHintWriteRouteOnly(sqlStatementContext) || isAllowWriteDataSourceQuery();
}
private boolean isWriteRouteStatement(final SQLStatementContext<?> sqlStatementContext) {
@@ -68,4 +69,9 @@ public final class ReadwriteSplittingDataSourceRouter {
private boolean isHintWriteRouteOnly(final SQLStatementContext<?> sqlStatementContext) {
return HintManager.isWriteRouteOnly() || (sqlStatementContext instanceof CommonSQLStatementContext && ((CommonSQLStatementContext<?>) sqlStatementContext).isHintWriteRouteOnly());
}
+
+ private boolean isAllowWriteDataSourceQuery() {
+ return rule.getEnabledReplicaDataSources().isEmpty() && (rule.getReadwriteSplittingStrategy() instanceof DynamicReadwriteSplittingStrategy)
+ && ((DynamicReadwriteSplittingStrategy) rule.getReadwriteSplittingStrategy()).isAllowWriteDataSourceQuery();
+ }
}
diff --git a/shardingsphere-features/shardingsphere-readwrite-splitting/shardingsphere-readwrite-splitting-core/src/main/java/org/apache/shardingsphere/readwritesplitting/strategy/ReadwriteSplittingStrategyFactory.java b/shardingsphere-features/shardingsphere-readwrite-splitting/shardingsphere-readwrite-splitting-core/src/main/java/org/apache/shardingsphere/readwritesplitting/strategy/ReadwriteSplittingStrategyFactory.java
index 92300a7df12..26e44c296f8 100644
--- a/shardingsphere-features/shardingsphere-readwrite-splitting/shardingsphere-readwrite-splitting-core/src/main/java/org/apache/shardingsphere/readwritesplitting/strategy/ReadwriteSplittingStrategyFactory.java
+++ b/shardingsphere-features/shardingsphere-readwrite-splitting/shardingsphere-readwrite-splitting-core/src/main/java/org/apache/shardingsphere/readwritesplitting/strategy/ReadwriteSplittingStrategyFactory.java
@@ -61,6 +61,7 @@ public final class ReadwriteSplittingStrategyFactory {
Preconditions.checkArgument(!Strings.isNullOrEmpty(autoAwareDataSourceName), "Auto aware data source name is required.");
Optional<DynamicDataSourceStrategy> dynamicDataSourceStrategy = DynamicDataSourceStrategyFactory.findInstance();
Preconditions.checkArgument(dynamicDataSourceStrategy.isPresent(), "Dynamic data source strategy is required.");
- return new DynamicReadwriteSplittingStrategy(autoAwareDataSourceName, dynamicDataSourceStrategy.get());
+ boolean allowWriteDataSourceQuery = Boolean.parseBoolean(props.getOrDefault("write-data-source-query-enabled", String.valueOf(Boolean.TRUE)).toString());
+ return new DynamicReadwriteSplittingStrategy(autoAwareDataSourceName, allowWriteDataSourceQuery, dynamicDataSourceStrategy.get());
}
}
diff --git a/shardingsphere-features/shardingsphere-readwrite-splitting/shardingsphere-readwrite-splitting-core/src/main/java/org/apache/shardingsphere/readwritesplitting/strategy/type/DynamicReadwriteSplittingStrategy.java b/shardingsphere-features/shardingsphere-readwrite-splitting/shardingsphere-readwrite-splitting-core/src/main/java/org/apache/shardingsphere/readwritesplitting/strategy/type/DynamicReadwriteSplittingStrategy.java
index 13960979619..15694de2cce 100644
--- a/shardingsphere-features/shardingsphere-readwrite-splitting/shardingsphere-readwrite-splitting-core/src/main/java/org/apache/shardingsphere/readwritesplitting/strategy/type/DynamicReadwriteSplittingStrategy.java
+++ b/shardingsphere-features/shardingsphere-readwrite-splitting/shardingsphere-readwrite-splitting-core/src/main/java/org/apache/shardingsphere/readwritesplitting/strategy/type/DynamicReadwriteSplittingStrategy.java
@@ -36,6 +36,8 @@ public final class DynamicReadwriteSplittingStrategy implements ReadwriteSplitti
private final String autoAwareDataSourceName;
+ private final boolean allowWriteDataSourceQuery;
+
private final DynamicDataSourceStrategy dynamicDataSourceStrategy;
@Override
diff --git a/shardingsphere-features/shardingsphere-readwrite-splitting/shardingsphere-readwrite-splitting-core/src/test/java/org/apache/shardingsphere/readwritesplitting/rule/ReadWriteSplittingDataSourceRuleConfigurationTest.java b/shardingsphere-features/shardingsphere-readwrite-splitting/shardingsphere-readwrite-splitting-core/src/test/java/org/apache/shardingsphere/readwritesplitting/rule/ReadWriteSplittingDataSourceRuleConfigurationTest.java
index 58633681d68..8365645949d 100644
--- a/shardingsphere-features/shardingsphere-readwrite-splitting/shardingsphere-readwrite-splitting-core/src/test/java/org/apache/shardingsphere/readwritesplitting/rule/ReadWriteSplittingDataSourceRuleConfigurationTest.java
+++ b/shardingsphere-features/shardingsphere-readwrite-splitting/shardingsphere-readwrite-splitting-core/src/test/java/org/apache/shardingsphere/readwritesplitting/rule/ReadWriteSplittingDataSourceRuleConfigurationTest.java
@@ -24,7 +24,7 @@ import org.junit.Test;
import java.util.Properties;
import static org.hamcrest.CoreMatchers.is;
-import static org.junit.Assert.assertNull;
+import static org.junit.Assert.assertNotNull;
import static org.junit.Assert.assertThat;
public final class ReadWriteSplittingDataSourceRuleConfigurationTest {
@@ -35,13 +35,15 @@ public final class ReadWriteSplittingDataSourceRuleConfigurationTest {
@Before
public void setup() {
- readwriteSplittingDataSourceRuleConfig = new ReadwriteSplittingDataSourceRuleConfiguration("ds", "Static", getProperties("write_ds", "read_ds_0,read_ds_1"), "");
- readwriteSplittingDataSourceRuleConfigDynamic = new ReadwriteSplittingDataSourceRuleConfiguration("ds", "Dynamic", getProperties("write_ds", "read_ds_0,read_ds_1"), "");
+ readwriteSplittingDataSourceRuleConfig = new ReadwriteSplittingDataSourceRuleConfiguration("ds", "Static", getStaticReadwriteSplittingProperties(), "");
+ readwriteSplittingDataSourceRuleConfigDynamic = new ReadwriteSplittingDataSourceRuleConfiguration("ds", "Dynamic", getDynamicReadwriteSplittingProperties(), "");
}
@Test
- public void assertGetAutoAwareDataSourceName() {
- assertNull(readwriteSplittingDataSourceRuleConfigDynamic.getProps().getProperty("auto-aware-data-source-name"));
+ public void assertDynamicReadWriteSplittingConfig() {
+ assertNotNull(readwriteSplittingDataSourceRuleConfigDynamic.getProps());
+ assertThat(readwriteSplittingDataSourceRuleConfigDynamic.getProps().getProperty("auto-aware-data-source-name"), is("readwrite_ds"));
+ assertThat(readwriteSplittingDataSourceRuleConfigDynamic.getProps().getProperty("write-data-source-query-enabled"), is("false"));
}
@Test
@@ -54,10 +56,17 @@ public final class ReadWriteSplittingDataSourceRuleConfigurationTest {
assertThat(readwriteSplittingDataSourceRuleConfig.getProps().getProperty("read-data-source-names"), is("read_ds_0,read_ds_1"));
}
- private Properties getProperties(final String writeDataSource, final String readDataSources) {
+ private Properties getStaticReadwriteSplittingProperties() {
Properties result = new Properties();
- result.setProperty("write-data-source-name", writeDataSource);
- result.setProperty("read-data-source-names", readDataSources);
+ result.setProperty("write-data-source-name", "write_ds");
+ result.setProperty("read-data-source-names", "read_ds_0,read_ds_1");
+ return result;
+ }
+
+ private Properties getDynamicReadwriteSplittingProperties() {
+ Properties result = new Properties();
+ result.setProperty("auto-aware-data-source-name", "readwrite_ds");
+ result.setProperty("write-data-source-query-enabled", "false");
return result;
}
}