You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2020/01/12 13:31:41 UTC

[GitHub] [incubator-shardingsphere] kimmking opened a new pull request #3945: fix #3929 support range sharding to broadcast sql to all actual tables

kimmking opened a new pull request #3945: fix #3929 support range sharding to broadcast sql to all actual tables
URL: https://github.com/apache/incubator-shardingsphere/pull/3945
 
 
   Fixes #3929
   
   Changes proposed in this pull request:
   - support range sharding(SQL with ‘>’, '<' and between...and...)
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-shardingsphere] terrymanu edited a comment on issue #3945: fix #3929 support range sharding to broadcast sql to all actual tables

Posted by GitBox <gi...@apache.org>.
terrymanu edited a comment on issue #3945: fix #3929 support range sharding to broadcast sql to all actual tables
URL: https://github.com/apache/incubator-shardingsphere/pull/3945#issuecomment-573419900
 
 
   We need to discuss this pull request:
   1. For standard sharding strategy, user need config precise algorithm (for =, IN) and range algorithm (for BETWEEN AND, <, >,  >=, <=). If range algorithm absent, the SQL contains BETWEEN should query all data nodes. That is correct, because user know they do not config range algorithm explicitly;
   2. For inline sharding strategy, user only can config precise algorithm via inline expression, and cannot config range algorithm. So throw exception when SQL contains BETWEEN may better than query all data nodes. If query all data nodes instead of explicit exception, user may missing the potential performance issue.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-shardingsphere] kimmking commented on issue #3945: fix #3929 support range sharding to broadcast sql to all actual tables

Posted by GitBox <gi...@apache.org>.
kimmking commented on issue #3945: fix #3929 support range sharding to broadcast sql to all actual tables
URL: https://github.com/apache/incubator-shardingsphere/pull/3945#issuecomment-573430784
 
 
   I found this error is puzzle for me in this case in past 3 weeks.
   
   There is a sharding table named 'test' with 4 actual tables with sharding key is 'id' column, then we will try proxy with:
   - 1.select * from test
   - 2.select * from test where id=102
   - 3.select * from test where id>102
   - 4.select * from test where id>102 and id<104
   - 5.select * from test where id<>102
   - 6.select * from test where id!=102
   - 7.select * from test where id between 102 and 104
   - 8.select * from test where id in (102, 104)
   - 9.select * from test where id not in (102, 104)
   - 10.select * from test where id>102 or id<102
   
   Executing these No. 3\4\7\10 SQL will occur the same exception：
   ```
   [2020-01-12 14:01:13] [C1000][10002] 2Unknown exception: [Inline strategy cannot support range sharding.]
   ```
   
   When I read `incubator-shardingsphere/sharding-core/sharding-core-common/src/main/java/org/apache/shardingsphere/core/strategy/route/inline/InlineShardingStrategy.java`, we can learn InlineShardingStrategy isn't support Range query: 
   ```
   Preconditions.checkState(shardingValue instanceof ListRouteValue, "Inline strategy cannot support range sharding.");  
   ```
   
   But the puzzle is: the No.10 SQL(can't executed) is equals with the No.5(executed correctly), the I wanna ask why and how to support range query in sharding strategy？
   
   Aside from performance considerations, we can find the most effective way to directly support this feature by broadcasting the whole Range query to all sharding db/tables，just like  ‘<>’ operator.
   
   
   Then we can let it work well and optimized later，we can insert here one line to broadcasting sql：
   ```
   if (shardingValue instanceof RangeRouteValue) {
               return availableTargetNames;
           }
   ```
   
   Then the No.3\4\7\10 SQL will be executed correctly.
   
   Related PR： https://github.com/apache/incubator-shardingsphere/pull/3945
   
   ------------above from issue 3929 ----------
   
   PS:  this two sql is supported now:
   - "select * from test where id<>102"
   - "select * from test where id not in (102, 104)"
   
   I guess they're also range query and have a lower performance than the condition "id>102".

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-shardingsphere] terrymanu merged pull request #3945: fix #3929 support range sharding to broadcast sql to all actual tables

Posted by GitBox <gi...@apache.org>.
terrymanu merged pull request #3945: fix #3929 support range sharding to broadcast sql to all actual tables
URL: https://github.com/apache/incubator-shardingsphere/pull/3945
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-shardingsphere] kimmking commented on issue #3945: fix #3929 support range sharding to broadcast sql to all actual tables

Posted by GitBox <gi...@apache.org>.
kimmking commented on issue #3945: fix #3929 support range sharding to broadcast sql to all actual tables
URL: https://github.com/apache/incubator-shardingsphere/pull/3945#issuecomment-577061879
 
 
   Add a property named `allow.range.query.with.inline.sharding` to allow range query with inline sharding, default false.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-shardingsphere] terrymanu commented on issue #3945: fix #3929 support range sharding to broadcast sql to all actual tables

Posted by GitBox <gi...@apache.org>.
terrymanu commented on issue #3945: fix #3929 support range sharding to broadcast sql to all actual tables
URL: https://github.com/apache/incubator-shardingsphere/pull/3945#issuecomment-573646586
 
 
   We can consider about add a new prop key to let user define whether full query for range query or throw exception.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-shardingsphere] terrymanu commented on issue #3945: fix #3929 support range sharding to broadcast sql to all actual tables

Posted by GitBox <gi...@apache.org>.
terrymanu commented on issue #3945: fix #3929 support range sharding to broadcast sql to all actual tables
URL: https://github.com/apache/incubator-shardingsphere/pull/3945#issuecomment-573419900
 
 
   We need to discuss this change:
   1. For standard sharding strategy, user need config precise algorithm (for =, IN) and range algorithm (for BETWEEN AND, <, >,  >=, <=). If user do not config range algorithm, the BETWEEN should query all data nodes, that correct;
   2. For inline sharding strategy, user only can config precise algorithm via inline expression, and cannot config range algorithm. So throw exception when SQL contains BETWEEN may better than query all data nodes.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-shardingsphere] kimmking commented on issue #3945: fix #3929 support range sharding to broadcast sql to all actual tables

Posted by GitBox <gi...@apache.org>.
kimmking commented on issue #3945: fix #3929 support range sharding to broadcast sql to all actual tables
URL: https://github.com/apache/incubator-shardingsphere/pull/3945#issuecomment-577097007
 
 
   /ci

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-shardingsphere] terrymanu edited a comment on issue #3945: fix #3929 support range sharding to broadcast sql to all actual tables

Posted by GitBox <gi...@apache.org>.
terrymanu edited a comment on issue #3945: fix #3929 support range sharding to broadcast sql to all actual tables
URL: https://github.com/apache/incubator-shardingsphere/pull/3945#issuecomment-573419900
 
 
   We need to discuss this pull request:
   1. For standard sharding strategy, user need config precise algorithm (for =, IN) and range algorithm (for BETWEEN AND, <, >,  >=, <=). If user do not config range algorithm, the BETWEEN should query all data nodes, that correct;
   2. For inline sharding strategy, user only can config precise algorithm via inline expression, and cannot config range algorithm. So throw exception when SQL contains BETWEEN may better than query all data nodes.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services