You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by "Fottas (via GitHub)" <gi...@apache.org> on 2025/09/02 02:59:33 UTC

Re: [PR] support IN clause optimization for sharding key [shardingsphere]

Fottas commented on PR #36370:
URL: https://github.com/apache/shardingsphere/pull/36370#issuecomment-3243630955

   > Hi @Fottas, this PR looks very complicated. Could you please submit an issue first to describe your design?
   
   Hi @strongduanmu:
   Issue Description: IN Predicate Sharding Optimization
   
   Problem: Currently ShardingSphere sends all values in IN predicates to every shard, causing unnecessary data transfer and computation. For example, WHERE id IN (1,2,3,4) is sent as-is to all shards, even though based on sharding keys we can determine that values 1,3 belong to shard A while values 2,4 belong to shard B.
   Solution: Route-aware IN predicate splitting that distributes values only to their target shards based on sharding algorithms.
   
   Technical Design
   Implementation Principle: Parse each value in IN expressions, invoke existing sharding algorithms to calculate target shards, then rewrite IN clauses for each RouteUnit to contain only values belonging to that shard.
   
   Core Components:
   1. ShardingInPredicateValue - Encapsulates IN values with their target route information
   2. ShardingInPredicateToken - SQL token that rewrites IN clauses per route, filtering values and parameters
   3. ShardingInPredicateTokenGenerator - Analyzes IN expressions and calculates shard distribution using existing sharding 
   algorithms
   4.ParameterFilterable - Cross-module decoupling interface that enables parameter filtering capability between modules, since ShardingInPredicateToken is in the sharding module while RouteSQLRewriteEngine is in the infra module
   
   Key Features:
   - Supports both standard and complex sharding strategies
   - Handles parameter markers and literals
   - Optimizes single-value cases to equality comparisons
   - Achieves module decoupling through interface abstraction, allowing core engine to work without direct dependencies on - sharding token implementations


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org