You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2021/11/08 06:10:31 UTC

[GitHub] [shardingsphere] strongduanmu commented on a change in pull request #13486: fixbug: Support not nested subquery (from Projections, TableSegment and Expression) on SelectStatement

strongduanmu commented on a change in pull request #13486:
URL: https://github.com/apache/shardingsphere/pull/13486#discussion_r744423372



##########
File path: shardingsphere-features/shardingsphere-encrypt/shardingsphere-encrypt-core/src/main/java/org/apache/shardingsphere/encrypt/rewrite/condition/EncryptConditionEngine.java
##########
@@ -71,6 +73,26 @@
             SelectStatementContext selectStatementContext = ((InsertStatementContext) sqlStatementContext).getInsertSelectContext().getSelectStatementContext();
             result.addAll(createEncryptConditionsOnWhereSegment(selectStatementContext));
         }
+        if (sqlStatementContext instanceof SelectStatementContext) {
+            SelectStatementContext selectStatementContext = (SelectStatementContext) sqlStatementContext;

Review comment:
       @cheese8 Can we extract this logic into the method?

##########
File path: shardingsphere-features/shardingsphere-encrypt/shardingsphere-encrypt-core/src/main/java/org/apache/shardingsphere/encrypt/rewrite/token/generator/impl/EncryptPredicateColumnTokenGenerator.java
##########
@@ -61,28 +62,44 @@
     @Override
     protected boolean isGenerateSQLTokenForEncrypt(final SQLStatementContext sqlStatementContext) {
         return (sqlStatementContext instanceof WhereAvailable && ((WhereAvailable) sqlStatementContext).getWhere().isPresent())
-            || (sqlStatementContext instanceof SelectStatementContext && ((SelectStatementContext) sqlStatementContext).isContainsJoinQuery())
-            || ((sqlStatementContext instanceof InsertStatementContext) && null != ((InsertStatementContext) sqlStatementContext).getInsertSelectContext());
+                || (sqlStatementContext instanceof SelectStatementContext && (((SelectStatementContext) sqlStatementContext).isContainsJoinQuery()
+                || ((SelectStatementContext) sqlStatementContext).isContainsSubquery()))
+                || ((sqlStatementContext instanceof InsertStatementContext) && null != ((InsertStatementContext) sqlStatementContext).getInsertSelectContext());
     }
     
     @Override
     public Collection<SubstitutableColumnNameToken> generateSQLTokens(final SQLStatementContext sqlStatementContext) {
         Collection<WhereSegment> whereSegments = getWhereSegments(sqlStatementContext);
         Collection<AndPredicate> andPredicates = whereSegments.stream().flatMap(each -> ExpressionExtractUtil.getAndPredicates(each.getExpr()).stream()).collect(Collectors.toList());
         Map<String, String> columnTableNames = getColumnTableNames(sqlStatementContext, andPredicates, whereSegments);
-        return andPredicates.stream().flatMap(each -> generateSQLTokens(each.getPredicates(), columnTableNames).stream()).collect(Collectors.toCollection(LinkedHashSet::new));
+        return andPredicates.stream().flatMap(each -> generateSQLTokens(sqlStatementContext, each.getPredicates(), columnTableNames).stream()).collect(Collectors.toCollection(LinkedHashSet::new));
     }
-    
-    private Collection<SubstitutableColumnNameToken> generateSQLTokens(final Collection<ExpressionSegment> predicates, final Map<String, String> columnTableNames) {
+
+    private Collection<SubstitutableColumnNameToken> generateSQLTokens(final SQLStatementContext sqlStatementContext, final Collection<ExpressionSegment> predicates,
+                                                                       final Map<String, String> columnTableNames) {
         Collection<SubstitutableColumnNameToken> result = new LinkedList<>();
+        Map<String, Map<String, Map<String, String>>> rewriteMetaDataMap = new HashMap<>();
+        if (sqlStatementContext instanceof SelectStatementContext) {
+            rewriteMetaDataMap = ((SelectStatementContext) sqlStatementContext).getRewriteMetaDataMap();
+            if (rewriteMetaDataMap == null) {

Review comment:
       @cheese8 Please put null on the left side of the expression.

##########
File path: shardingsphere-features/shardingsphere-encrypt/shardingsphere-encrypt-core/src/main/java/org/apache/shardingsphere/encrypt/rewrite/token/generator/impl/EncryptProjectionTokenGenerator.java
##########
@@ -67,63 +71,96 @@ protected boolean isGenerateSQLTokenForEncrypt(final SQLStatementContext sqlStat
     @Override
     public Collection<SubstitutableColumnNameToken> generateSQLTokens(final SQLStatementContext sqlStatementContext) {
         Collection<SubstitutableColumnNameToken> result = new LinkedHashSet<>();
+        Map<String, Map<String, Map<String, String>>> rewriteMetaDataMap = new HashMap<>();
         if (sqlStatementContext instanceof InsertStatementContext) {
-            result.addAll(generateSQLTokens(((InsertStatementContext) sqlStatementContext).getInsertSelectContext().getSelectStatementContext(), true));
+            result.addAll(generateSQLTokens(((InsertStatementContext) sqlStatementContext).getInsertSelectContext().getSelectStatementContext(), Optional.empty(), SubqueryEnum.InsertSelectSubquery,
+                    rewriteMetaDataMap));
         }
         if (sqlStatementContext instanceof SelectStatementContext) {
-            result.addAll(generateSQLTokens((SelectStatementContext) sqlStatementContext, false));
+            SelectStatementContext selectStatementContext = (SelectStatementContext) sqlStatementContext;
+            if (selectStatementContext.isContainsSubquery()) {
+                SubqueryExtractUtil.getSubqueryTableSegmentsFromTableSegment(selectStatementContext.getSqlStatement().getFrom()).forEach(each -> result.addAll(generateSQLTokens(
+                        new SelectStatementContext(selectStatementContext.getMetaDataMap(), selectStatementContext.getParameters(), each.getSubquery().getSelect(),
+                                selectStatementContext.getSchemaName()), each.getAlias(), SubqueryEnum.NestedProjectionTabsegmentSubquery, rewriteMetaDataMap)));
+                SubqueryExtractUtil.getSubquerySegmentsFromProjections(selectStatementContext.getSqlStatement().getProjections()).forEach(each -> result.addAll(generateSQLTokens(
+                        new SelectStatementContext(selectStatementContext.getMetaDataMap(), selectStatementContext.getParameters(), each.getSelect(),
+                                selectStatementContext.getSchemaName()), Optional.empty(), SubqueryEnum.ProjectionSubqery, rewriteMetaDataMap)));
+                selectStatementContext.getWhere().ifPresent(where -> SubqueryExtractUtil.getSubquerySegmentsFromExpression(where.getExpr()).forEach(each -> result.addAll(generateSQLTokens(
+                        new SelectStatementContext(selectStatementContext.getMetaDataMap(), selectStatementContext.getParameters(), each.getSelect(),
+                                selectStatementContext.getSchemaName()), Optional.empty(), SubqueryEnum.ExpressionSubqery, rewriteMetaDataMap))));
+            }
+            result.addAll(generateSQLTokens(selectStatementContext, Optional.empty(), SubqueryEnum.None, rewriteMetaDataMap));
         }
         return result;
     }
-    
-    private Collection<SubstitutableColumnNameToken> generateSQLTokens(final SelectStatementContext selectStatementContext, final boolean insertSelect) {
+
+    private Collection<SubstitutableColumnNameToken> generateSQLTokens(final SelectStatementContext selectStatementContext, final Optional<String> alias,
+                                                                       final SubqueryEnum subqueryEnum, final Map<String, Map<String, Map<String, String>>> rewriteMetaDataMap) {
         Collection<SubstitutableColumnNameToken> result = new LinkedHashSet<>();
         ProjectionsSegment projectionsSegment = selectStatementContext.getSqlStatement().getProjections();
         for (String each : selectStatementContext.getTablesContext().getTableNames()) {
-            getEncryptRule().findEncryptTable(each).map(optional -> generateSQLTokens(projectionsSegment, each, selectStatementContext, optional, insertSelect)).ifPresent(result::addAll);
+            getEncryptRule().findEncryptTable(each).map(optional -> generateSQLTokens(projectionsSegment, each, selectStatementContext, optional, alias, subqueryEnum,
+                    rewriteMetaDataMap)).ifPresent(result::addAll);
         }
+        selectStatementContext.setRewriteMetaDataMap(rewriteMetaDataMap);

Review comment:
       Why set rewriteMetaDataMap to Context? The value in the Context should be completed when the Context is initialized.

##########
File path: shardingsphere-features/shardingsphere-encrypt/shardingsphere-encrypt-core/src/main/java/org/apache/shardingsphere/encrypt/rewrite/token/generator/impl/EncryptPredicateColumnTokenGenerator.java
##########
@@ -61,28 +62,44 @@
     @Override
     protected boolean isGenerateSQLTokenForEncrypt(final SQLStatementContext sqlStatementContext) {
         return (sqlStatementContext instanceof WhereAvailable && ((WhereAvailable) sqlStatementContext).getWhere().isPresent())
-            || (sqlStatementContext instanceof SelectStatementContext && ((SelectStatementContext) sqlStatementContext).isContainsJoinQuery())
-            || ((sqlStatementContext instanceof InsertStatementContext) && null != ((InsertStatementContext) sqlStatementContext).getInsertSelectContext());
+                || (sqlStatementContext instanceof SelectStatementContext && (((SelectStatementContext) sqlStatementContext).isContainsJoinQuery()
+                || ((SelectStatementContext) sqlStatementContext).isContainsSubquery()))
+                || ((sqlStatementContext instanceof InsertStatementContext) && null != ((InsertStatementContext) sqlStatementContext).getInsertSelectContext());
     }
     
     @Override
     public Collection<SubstitutableColumnNameToken> generateSQLTokens(final SQLStatementContext sqlStatementContext) {
         Collection<WhereSegment> whereSegments = getWhereSegments(sqlStatementContext);
         Collection<AndPredicate> andPredicates = whereSegments.stream().flatMap(each -> ExpressionExtractUtil.getAndPredicates(each.getExpr()).stream()).collect(Collectors.toList());
         Map<String, String> columnTableNames = getColumnTableNames(sqlStatementContext, andPredicates, whereSegments);
-        return andPredicates.stream().flatMap(each -> generateSQLTokens(each.getPredicates(), columnTableNames).stream()).collect(Collectors.toCollection(LinkedHashSet::new));
+        return andPredicates.stream().flatMap(each -> generateSQLTokens(sqlStatementContext, each.getPredicates(), columnTableNames).stream()).collect(Collectors.toCollection(LinkedHashSet::new));
     }
-    
-    private Collection<SubstitutableColumnNameToken> generateSQLTokens(final Collection<ExpressionSegment> predicates, final Map<String, String> columnTableNames) {
+
+    private Collection<SubstitutableColumnNameToken> generateSQLTokens(final SQLStatementContext sqlStatementContext, final Collection<ExpressionSegment> predicates,
+                                                                       final Map<String, String> columnTableNames) {
         Collection<SubstitutableColumnNameToken> result = new LinkedList<>();
+        Map<String, Map<String, Map<String, String>>> rewriteMetaDataMap = new HashMap<>();
+        if (sqlStatementContext instanceof SelectStatementContext) {
+            rewriteMetaDataMap = ((SelectStatementContext) sqlStatementContext).getRewriteMetaDataMap();
+            if (rewriteMetaDataMap == null) {
+                rewriteMetaDataMap = new HashMap<>();
+            }
+        }
         for (ExpressionSegment each : predicates) {
             for (ColumnSegment column : ColumnExtractor.extract(each)) {
+                int startIndex = column.getOwner().isPresent() ? column.getOwner().get().getStopIndex() + 2 : column.getStartIndex();
+                int stopIndex = column.getStopIndex();
+                if (!rewriteMetaDataMap.isEmpty()) {
+                    Map<String, Map<String, String>> value = rewriteMetaDataMap.get(column.getOwner().get().getIdentifier().getValue());

Review comment:
       @cheese8 Can a meaningful name be used instead of value?

##########
File path: shardingsphere-features/shardingsphere-encrypt/shardingsphere-encrypt-core/src/main/java/org/apache/shardingsphere/encrypt/rewrite/token/generator/impl/EncryptProjectionTokenGenerator.java
##########
@@ -207,7 +316,15 @@ private ShorthandProjection getShorthandProjection(final ShorthandProjectionSegm
         }
         throw new IllegalStateException(String.format("Can not find shorthand projection segment, owner is: `%s`", owner.orElse(null)));
     }
-    
+
+    enum SubqueryEnum {

Review comment:
       @cheese8 Please extract the enum into a separate class.

##########
File path: shardingsphere-infra/shardingsphere-infra-binder/src/main/java/org/apache/shardingsphere/infra/binder/statement/dml/SelectStatementContext.java
##########
@@ -80,8 +81,17 @@
     
     private final String schemaName;
 
+    private final Map<String, ShardingSphereMetaData> metaDataMap;
+
+    private final List<Object> parameters;
+
+    @Setter
+    private Map<String, Map<String, Map<String, String>>> rewriteMetaDataMap;

Review comment:
       What is the purpose of adding these variables?

##########
File path: shardingsphere-features/shardingsphere-encrypt/shardingsphere-encrypt-core/src/main/java/org/apache/shardingsphere/encrypt/rewrite/token/generator/impl/EncryptPredicateColumnTokenGenerator.java
##########
@@ -52,7 +52,8 @@
  * Predicate column token generator for encrypt.
  */
 @Setter
-public final class EncryptPredicateColumnTokenGenerator extends BaseEncryptSQLTokenGenerator implements CollectionSQLTokenGenerator, SchemaMetaDataAware, QueryWithCipherColumnAware {
+public final class EncryptPredicateColumnTokenGenerator extends BaseEncryptSQLTokenGenerator implements CollectionSQLTokenGenerator, SchemaMetaDataAware,
+        QueryWithCipherColumnAware {

Review comment:
       Please keep the original code format.

##########
File path: shardingsphere-features/shardingsphere-encrypt/shardingsphere-encrypt-core/src/main/java/org/apache/shardingsphere/encrypt/rewrite/token/generator/impl/EncryptPredicateColumnTokenGenerator.java
##########
@@ -103,23 +120,42 @@ protected boolean isGenerateSQLTokenForEncrypt(final SQLStatementContext sqlStat
     
     private Collection<WhereSegment> getWhereSegments(final SQLStatementContext<?> sqlStatementContext) {
         Collection<WhereSegment> result = new LinkedList<>();
-        if (sqlStatementContext instanceof WhereAvailable && ((WhereAvailable) sqlStatementContext).getWhere().isPresent()) {
-            result.add(((WhereAvailable) sqlStatementContext).getWhere().get());
-        }
-        if (sqlStatementContext instanceof SelectStatementContext && ((SelectStatementContext) sqlStatementContext).isContainsJoinQuery()) {
-            result.addAll(WhereExtractUtil.getJoinWhereSegments((SelectStatement) sqlStatementContext.getSqlStatement()));
+        result.addAll(getWhereSegmentsFromWhereAvailable(sqlStatementContext));
+        if (sqlStatementContext instanceof SelectStatementContext) {
+            SelectStatementContext selectStatementContext = (SelectStatementContext) sqlStatementContext;
+            result.addAll(getWhereSegmentsOnJoinQuery(selectStatementContext));
+            if (selectStatementContext.isContainsSubquery()) {
+                selectStatementContext.getSubquerySegments().forEach(each -> {

Review comment:
       Please extract methods to avoid too deep code levels.

##########
File path: shardingsphere-features/shardingsphere-encrypt/shardingsphere-encrypt-core/src/main/java/org/apache/shardingsphere/encrypt/rewrite/token/generator/impl/EncryptProjectionTokenGenerator.java
##########
@@ -67,63 +71,96 @@ protected boolean isGenerateSQLTokenForEncrypt(final SQLStatementContext sqlStat
     @Override
     public Collection<SubstitutableColumnNameToken> generateSQLTokens(final SQLStatementContext sqlStatementContext) {
         Collection<SubstitutableColumnNameToken> result = new LinkedHashSet<>();
+        Map<String, Map<String, Map<String, String>>> rewriteMetaDataMap = new HashMap<>();
         if (sqlStatementContext instanceof InsertStatementContext) {
-            result.addAll(generateSQLTokens(((InsertStatementContext) sqlStatementContext).getInsertSelectContext().getSelectStatementContext(), true));
+            result.addAll(generateSQLTokens(((InsertStatementContext) sqlStatementContext).getInsertSelectContext().getSelectStatementContext(), Optional.empty(), SubqueryEnum.InsertSelectSubquery,
+                    rewriteMetaDataMap));
         }
         if (sqlStatementContext instanceof SelectStatementContext) {
-            result.addAll(generateSQLTokens((SelectStatementContext) sqlStatementContext, false));
+            SelectStatementContext selectStatementContext = (SelectStatementContext) sqlStatementContext;
+            if (selectStatementContext.isContainsSubquery()) {
+                SubqueryExtractUtil.getSubqueryTableSegmentsFromTableSegment(selectStatementContext.getSqlStatement().getFrom()).forEach(each -> result.addAll(generateSQLTokens(

Review comment:
       @cheese8 Please extract methods to avoid repeated looping code.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org