You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2022/12/01 12:05:02 UTC
[GitHub] [shardingsphere] theodoretsai opened a new issue, #22572: Ussue with IN statements Implementing ComplexKeysShardingAlgorithm
theodoretsai opened a new issue, #22572:
URL: https://github.com/apache/shardingsphere/issues/22572
I'm trying to implement the ComplexKeyShardingAlgorithm and I'm having an issue with IN statement queries when only a partial match is found.
For example for the routing configuration as following (Keeping it simple with one keyed column):
Default Datasource = DS1
Default Table = Table1
Strategy 1: Value A -> DS1.table1
Strategy 2: Value B -> DS1.table2
Strategy 3: Value C -> DS2.table1
Strategy 4: Value D -> DS2.table2
with the following query:
```
SELECT *
FROM logical_table
WHERE column = C OR column = D
```
Th Sharding Result should be :
`DS2.table1 because it C is routed there by stratedy 3`
`DS1.table1 because D doesn't match any strategyand default is selected`
now if I execute the query like this
```
SELECT *
FROM logical_table
WHERE column IN (C, D)
```
how am I supposed to get the same result of the OR statement?
This is the code from the StandardRoutingEngine:
```
private Collection<DataNode> route0(final TableRule tableRule, final List<RouteValue> databaseShardingValues,
final List<RouteValue> tableShardingValues) {
Collection<String> routedDataSources = routeDataSources(tableRule, databaseShardingValues);
Collection<DataNode> result = new LinkedList<>();
for (String each : routedDataSources) {
result.addAll(routeTables(tableRule, each, tableShardingValues));
}
return result;
}
```
When routing datasources it's pretty straitght forward, select the ones from strategies that match, and if there's sharding value that doesn't match any add the Default datasource as well.
Now when routing tables for each datasource it becomes tricky: when doing it with tables from a datasource match I should just return the table value that match, but how do I know when I am sharding table values of the default datasource which was selected because there was an unconsidered sharding value? (that no strategy matched)
Since I only have the tableShardingValues information available when implementing the doSharding method it's really hard to do so. I guess it is possible but would be too complicated, and probably not very friendly for performance, and would require some stateful events to be cached somewhere.
I hope I am missing something here and that there's a better way to do this, some help would be very appreciated.
Thank you very much.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] theodoretsai commented on issue #22572: Issue with IN statements Implementing ComplexKeysShardingAlgorithm (4.0.1)
Posted by GitBox <gi...@apache.org>.
theodoretsai commented on issue #22572:
URL: https://github.com/apache/shardingsphere/issues/22572#issuecomment-1336618820
I have checked out the code from the lastest version
org.apache.shardingsphere.sharding.route.engine.type.standard:
```
private Collection<String> routeDataSources(final TableRule tableRule, final ShardingStrategy databaseShardingStrategy, final List<ShardingConditionValue> databaseShardingValues) {
if (databaseShardingValues.isEmpty()) {
return tableRule.getActualDataSourceNames();
}
Collection<String> result = databaseShardingStrategy.doSharding(tableRule.getActualDataSourceNames(), databaseShardingValues, tableRule.getDataSourceDataNode(), props);
Preconditions.checkState(!result.isEmpty(), "No database route info");
Preconditions.checkState(tableRule.getActualDataSourceNames().containsAll(result),
"Some routed data sources do not belong to configured data sources. routed data sources: `%s`, configured data sources: `%s`", result, tableRule.getActualDataSourceNames());
return result;
}
private Collection<DataNode> routeTables(final TableRule tableRule, final String routedDataSource,
final ShardingStrategy tableShardingStrategy, final List<ShardingConditionValue> tableShardingValues) {
Collection<String> availableTargetTables = tableRule.getActualTableNames(routedDataSource);
Collection<String> routedTables = tableShardingValues.isEmpty()
? availableTargetTables
: tableShardingStrategy.doSharding(availableTargetTables, tableShardingValues, tableRule.getTableDataNode(), props);
Collection<DataNode> result = new LinkedList<>();
for (String each : routedTables) {
result.add(new DataNode(routedDataSource, each));
}
return result;
}
```
vs
4.0.1 org.apache.shardingsphere.core.route.type.standard:
```
private Collection<String> routeDataSources(final TableRule tableRule, final List<RouteValue> databaseShardingValues) {
if (databaseShardingValues.isEmpty()) {
return tableRule.getActualDatasourceNames();
}
Collection<String> result = new LinkedHashSet<>(shardingRule.getDatabaseShardingStrategy(tableRule).doSharding(tableRule.getActualDatasourceNames(), databaseShardingValues));
Preconditions.checkState(!result.isEmpty(), "no database route info");
Preconditions.checkState(tableRule.getActualDatasourceNames().containsAll(result),
"Some routed data sources do not belong to configured data sources. routed data sources: `%s`, configured data sources: `%s`", result, tableRule.getActualDatasourceNames());
return result;
}
private Collection<DataNode> routeTables(final TableRule tableRule, final String routedDataSource, final List<RouteValue> tableShardingValues) {
Collection<String> availableTargetTables = tableRule.getActualTableNames(routedDataSource);
Collection<String> routedTables = new LinkedHashSet<>(tableShardingValues.isEmpty() ? availableTargetTables
: shardingRule.getTableShardingStrategy(tableRule).doSharding(availableTargetTables, tableShardingValues));
Preconditions.checkState(!routedTables.isEmpty(), "no table route info");
Collection<DataNode> result = new LinkedList<>();
for (String each : routedTables) {
result.add(new DataNode(routedDataSource, each));
}
return result;
}
```
the routing logic seems to be the same, I haven't found a way to "communicate a partial match" for IN statements. Are there any changes elsewhere addressing this that I'm missing? If yes I can try to replicate the problem in the latest version.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] strongduanmu commented on issue #22572: Issue with IN statements Implementing ComplexKeysShardingAlgorithm (4.0.1)
Posted by GitBox <gi...@apache.org>.
strongduanmu commented on issue #22572:
URL: https://github.com/apache/shardingsphere/issues/22572#issuecomment-1336280000
Hi @theodoretsai, can you try the latest version? 4.x is too old.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Issue with IN statements Implementing ComplexKeysShardingAlgorithm (4.0.1) [shardingsphere]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #22572:
URL: https://github.com/apache/shardingsphere/issues/22572#issuecomment-2026018094
There hasn't been any activity on this issue recently, and in order to prioritize active issues, it will be marked as stale.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org