You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by "terrymanu (via GitHub)" <gi...@apache.org> on 2025/11/06 02:53:04 UTC
Re: [I] An Issue when working with the encryption feature [shardingsphere]
terrymanu commented on issue #36841:
URL: https://github.com/apache/shardingsphere/issues/36841#issuecomment-3494625332
Analysis of the Issue
This issue occurs when both encryption (with cipher and assistedQuery) and sharding features are enabled simultaneously. The problem is a SQL rewrite conflict between multiple feature modules, not a simple bug that can be easily fixed.
Root Cause Analysis
Why does this error only occur when using both encryption and sharding together?
The fundamental cause is SQL rewrite mechanism conflicts:
1. Sharding feature requirements:
- Needs to rewrite logical SQL to physical SQL
- Handles primary key generation (like Snowflake algorithm)
- Dynamically adds primary key columns and parameters to INSERT statements
2. Encryption feature requirements:
- Encrypts sensitive columns
- When assistedQuery is enabled, adds assisted query columns (like assisted_username)
- Adds new columns and corresponding encrypted parameters after original encrypted columns
3. Conflict occurs when:
- Both features need to rewrite the same INSERT statement
- Both need to add new columns and parameters
- When the original encrypted column is the last column in the table, conflicts occur in adding new column positions
- Column addition order and parameter addition order become inconsistent, causing misalignment
Why does sharding alone work without problems?
When using sharding alone, only the sharding feature needs to modify SQL, so there are no coordination issues between multiple feature modules. Column and parameter addition is single-threaded and does not cause order
conflicts.
Current ShardingSphere Design Limitations
Why is this complex scenario currently not supported?
1. Architectural Complexity
- ShardingSphere uses a plugin-based architecture where feature modules are relatively independent
- Multiple feature modules require complex coordination mechanisms when working together
- The current architecture does not fully consider multi-module collaborative SQL rewriting scenarios
2. Feature Combination Challenges
- Encryption and sharding combinations involve interactions at multiple levels
- SQL rewriting, parameter processing, and routing strategies all need coordination
- Requires a unified derived column management framework, which currently lacks such design
3. Development Resource Constraints
- ShardingSphere community has limited development resources
- Needs to prioritize core functionality stability
- Complex feature combination compatibility is a high priority but not the current focus
Community PR Analysis
Issues with PR #36899:
This PR attempts to solve the problem by modifying the Token sorting mechanism, but has clear limitations:
1. Treating symptoms, not root cause: It only adjusts sorting to prevent errors in specific scenarios, without fundamentally solving the architectural coordination problem
2. Hardcoded solution: Uses fixed sorting values, lacking flexibility and extensibility
3. Fragile approach: When other feature modules or more complex scenarios exist, this solution may fail or even introduce new problems
Conclusion
ShardingSphere currently does not support complex combinations of encryption and sharding features.
This is not a simple bug fix, but an important technical challenge involving ShardingSphere's core architectural design. It requires:
1. Architectural redesign: Establish unified SQL rewriting coordination mechanisms
2. Deep integration of feature modules: Redesign interaction methods between different feature modules
3. Extensive testing and validation: Ensure stability of various combination scenarios
Until the ShardingSphere community has sufficient development resources to refactor the related architecture, this complex scenario will not receive comprehensive support. We recommend users consider adjusting system
architecture or temporarily avoiding feature combination usage when encountering such scenarios.
Recommendation
For your current situation, consider these workarounds:
- Avoid placing encrypted columns as the last column in the table
- Temporarily disable assistedQuery feature when using both encryption and sharding
- Use separate database instances for different feature requirements
This limitation is documented and will be addressed in future architectural improvements.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org