You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@ignite.apache.org by GitBox <gi...@apache.org> on 2021/09/28 16:55:50 UTC
[GitHub] [ignite] vladErmakov07 opened a new pull request #9441: IGNITE-14358 GridSubqueryJoinOptimizer fixes and tests.
vladErmakov07 opened a new pull request #9441:
URL: https://github.com/apache/ignite/pull/9441
Thank you for submitting the pull request to the Apache Ignite.
In order to streamline the review of the contribution
we ask you to ensure the following steps have been taken:
### The Contribution Checklist
- [ ] There is a single JIRA ticket related to the pull request.
- [ ] The web-link to the pull request is attached to the JIRA ticket.
- [ ] The JIRA ticket has the _Patch Available_ state.
- [ ] The pull request body describes changes that have been made.
The description explains _WHAT_ and _WHY_ was made instead of _HOW_.
- [ ] The pull request title is treated as the final commit message.
The following pattern must be used: `IGNITE-XXXX Change summary` where `XXXX` - number of JIRA issue.
- [ ] A reviewer has been mentioned through the JIRA comments
(see [the Maintainers list](https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute#HowtoContribute-ReviewProcessandMaintainers))
- [ ] The pull request has been checked by the Teamcity Bot and
the `green visa` attached to the JIRA ticket (see [TC.Bot: Check PR](https://mtcga.gridgain.com/prs.html))
### Notes
- [How to Contribute](https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute)
- [Coding abbreviation rules](https://cwiki.apache.org/confluence/display/IGNITE/Abbreviation+Rules)
- [Coding Guidelines](https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines)
- [Apache Ignite Teamcity Bot](https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+Teamcity+Bot)
If you need any help, please email dev@ignite.apache.org or ask anу advice on http://asf.slack.com _#ignite_ channel.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] korlov42 commented on a change in pull request #9441: IGNITE-14358 GridSubqueryJoinOptimizer fixes and tests.
Posted by GitBox <gi...@apache.org>.
korlov42 commented on a change in pull request #9441:
URL: https://github.com/apache/ignite/pull/9441#discussion_r731569537
##########
File path: modules/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/GridSubqueryJoinOptimizer.java
##########
@@ -399,33 +420,95 @@ private static boolean pullOutSubQryFromTableList(
/**
* Remap all columns that satisfy the predicate such they be referred to the given table.
*
- * @param ast Tree where to search columns.
+ * @param parent Tree where to search columns.
+ * @param subSelect Tree where to search column aliases.
* @param colPred Collection predicate.
* @param tbl Table.
*/
- private static void remapColumns(GridSqlAst ast, Predicate<GridSqlColumn> colPred, GridSqlAlias tbl) {
+ private static void remapColumns(GridSqlAst parent, GridSqlAst subSelect, Predicate<GridSqlColumn> colPred, GridSqlAlias tbl) {
ASTNodeFinder colFinder = new ASTNodeFinder(
- ast,
- (p, c) -> c instanceof GridSqlColumn && colPred.test((GridSqlColumn)c)
- );
+ parent,
Review comment:
When I said that `you need to unwrap child's alias it that case.`, I mean you literally should unwrap the alias by `GridSqlAlias.unwrap()` method. So the only change required is like this:
```
if (aliasOrPred != null)
res.getEl().child(res.getIdx(), aliasOrPred.getEl().child(aliasOrPred.getIdx()));
```
become
```
if (aliasOrPred != null)
res.getEl().child(res.getIdx(), GridSqlAlias.unwrap(aliasOrPred.getEl().child(aliasOrPred.getIdx())));
```
##########
File path: modules/indexing/src/test/java/org/apache/ignite/internal/processors/query/h2/GridSubqueryJoinOptimizerSelfTest.java
##########
@@ -644,18 +803,36 @@ public void testOptimizationShouldNotBeApplied6() {
private void check(String sql, int expSelectClauses) {
optimizationEnabled(false);
- List<List<?>> exp = cache.query(new SqlFieldsQuery(sql)).getAll();
+ FieldsQueryCursor<List<?>> qry = cache.query(new SqlFieldsQuery(sql));
+
+ List<GridQueryFieldMetadata> expMetaList = ((QueryCursorEx<List<?>>)qry).fieldsMeta();
+
+ List<List<?>> exp = qry.getAll();
exp.sort(ROW_COMPARATOR);
optimizationEnabled(true);
- List<List<?>> act = cache.query(new SqlFieldsQuery(sql).setEnforceJoinOrder(true)).getAll();
+ FieldsQueryCursor<List<?>> optQry = cache.query(new SqlFieldsQuery(sql).setEnforceJoinOrder(true));
+
+ List<GridQueryFieldMetadata> actMetaList = ((QueryCursorEx<List<?>>)optQry).fieldsMeta();
+
+ List<List<?>> act = optQry.getAll();
act.sort(ROW_COMPARATOR);
Assert.assertEquals("Result set mismatch", exp, act);
+ Assert.assertEquals("Result set column size mismatch", expMetaList.size(), actMetaList.size());
Review comment:
In case this assertion fails you won't get sufficient information to get the root cause.What item is missing? Or is there an excess item?
It would be better to derive separate collections for names and types and compare them as we do above for result set.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] korlov42 commented on a change in pull request #9441: IGNITE-14358 GridSubqueryJoinOptimizer fixes and tests.
Posted by GitBox <gi...@apache.org>.
korlov42 commented on a change in pull request #9441:
URL: https://github.com/apache/ignite/pull/9441#discussion_r719519738
##########
File path: modules/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/GridSubqueryJoinOptimizer.java
##########
@@ -399,33 +424,59 @@ private static boolean pullOutSubQryFromTableList(
/**
* Remap all columns that satisfy the predicate such they be referred to the given table.
*
- * @param ast Tree where to search columns.
+ * @param parent Tree where to search columns.
+ * @param subSelect Tree where to search column aliases.
* @param colPred Collection predicate.
* @param tbl Table.
*/
- private static void remapColumns(GridSqlAst ast, Predicate<GridSqlColumn> colPred, GridSqlAlias tbl) {
+ private static void remapColumns(GridSqlAst parent, GridSqlAst subSelect, Predicate<GridSqlColumn> colPred, GridSqlAlias tbl) {
ASTNodeFinder colFinder = new ASTNodeFinder(
- ast,
+ parent,
(p, c) -> c instanceof GridSqlColumn && colPred.test((GridSqlColumn)c)
);
ASTNodeFinder.Result res;
while ((res = colFinder.findNext()) != null) {
GridSqlColumn oldCol = res.getEl().child(res.getIdx());
- res.getEl().child(
- res.getIdx(),
- new GridSqlColumn(
- oldCol.column(),
- tbl,
- oldCol.schema(),
- tbl.alias(),
- oldCol.columnName()
- )
- );
+ BiPredicate<GridSqlAst, GridSqlAst> constPred = (p, c) ->
+ c != null && c.getSQL().equals(oldCol.columnName());
+
+ BiPredicate<GridSqlAst, GridSqlAst> aliasPred = (p, c) ->
+ c instanceof GridSqlAlias && ((GridSqlAlias)c).alias().equals(oldCol.columnName());
+
+ ASTNodeFinder.Result aliasOrPred = findNode(subSelect, constPred.or(aliasPred));
+
+ if (aliasOrPred != null)
+ res.getEl().child(res.getIdx(), aliasOrPred.getEl().child(aliasOrPred.getIdx()));
+ else {
+ res.getEl().child(
+ res.getIdx(),
+ new GridSqlColumn(
+ oldCol.column(),
+ tbl,
+ oldCol.schema(),
+ tbl.alias(),
+ oldCol.columnName()
+ )
+ );
+ }
}
}
+ /**
+ * Searches for firxt node in AST tree according to the given parameters.
Review comment:
firxt -> first
##########
File path: modules/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/GridSubqueryJoinOptimizer.java
##########
@@ -301,6 +302,15 @@ private static boolean isSimpleSelect(GridSqlQuery subQry) {
if (aggFinder.findNext() != null)
return false;
+
+ ASTNodeFinder operationFinder = new ASTNodeFinder(
Review comment:
please throw a couple of lines describing the situation this check protect from
##########
File path: modules/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/GridSubqueryJoinOptimizer.java
##########
@@ -399,33 +424,59 @@ private static boolean pullOutSubQryFromTableList(
/**
* Remap all columns that satisfy the predicate such they be referred to the given table.
*
- * @param ast Tree where to search columns.
+ * @param parent Tree where to search columns.
+ * @param subSelect Tree where to search column aliases.
* @param colPred Collection predicate.
* @param tbl Table.
*/
- private static void remapColumns(GridSqlAst ast, Predicate<GridSqlColumn> colPred, GridSqlAlias tbl) {
+ private static void remapColumns(GridSqlAst parent, GridSqlAst subSelect, Predicate<GridSqlColumn> colPred, GridSqlAlias tbl) {
ASTNodeFinder colFinder = new ASTNodeFinder(
- ast,
+ parent,
(p, c) -> c instanceof GridSqlColumn && colPred.test((GridSqlColumn)c)
);
ASTNodeFinder.Result res;
while ((res = colFinder.findNext()) != null) {
GridSqlColumn oldCol = res.getEl().child(res.getIdx());
- res.getEl().child(
- res.getIdx(),
- new GridSqlColumn(
- oldCol.column(),
- tbl,
- oldCol.schema(),
- tbl.alias(),
- oldCol.columnName()
- )
- );
+ BiPredicate<GridSqlAst, GridSqlAst> constPred = (p, c) ->
+ c != null && c.getSQL().equals(oldCol.columnName());
+
+ BiPredicate<GridSqlAst, GridSqlAst> aliasPred = (p, c) ->
+ c instanceof GridSqlAlias && ((GridSqlAlias)c).alias().equals(oldCol.columnName());
+
+ ASTNodeFinder.Result aliasOrPred = findNode(subSelect, constPred.or(aliasPred));
+
+ if (aliasOrPred != null)
+ res.getEl().child(res.getIdx(), aliasOrPred.getEl().child(aliasOrPred.getIdx()));
Review comment:
In case the parent node is alias you render an invalid sql like `SELECT child_col AS childs_alias AS parent_alias FROM ...`, so you need to unwrap child's alias it that case.
Please add an appropriate test for this case.
##########
File path: modules/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/GridSubqueryJoinOptimizer.java
##########
@@ -383,11 +393,26 @@ private static boolean pullOutSubQryFromTableList(
else
target.child(childInd, subTbl);
- if (subSel.where() != null)
- parent.where(parent.where() == null ? subSel.where() : new GridSqlOperation(AND, parent.where(), subSel.where()));
+ GridSqlAst where = subSel.where();
+ if (where != null) {
+ ASTNodeFinder.Result joinNode = findNode(parent, (p, c) -> c instanceof GridSqlJoin);
Review comment:
you already have a `target`, which is either `null` means that the subquery is the only table in the list of tables, or `join`. So this whole branch could be reduced to something like this:
```
if (target != null) {
GridSqlJoin join = (GridSqlJoin)target;
join.child(GridSqlJoin.ON_CHILD, new GridSqlOperation(AND, join.on(), where));
}
else
parent.where(parent.where() == null ? where : new GridSqlOperation(AND, parent.where(), where));
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] Berkof commented on a change in pull request #9441: IGNITE-14358 GridSubqueryJoinOptimizer fixes and tests.
Posted by GitBox <gi...@apache.org>.
Berkof commented on a change in pull request #9441:
URL: https://github.com/apache/ignite/pull/9441#discussion_r731584440
##########
File path: modules/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/GridSubqueryJoinOptimizer.java
##########
@@ -383,11 +395,20 @@ private static boolean pullOutSubQryFromTableList(
else
target.child(childInd, subTbl);
- if (subSel.where() != null)
- parent.where(parent.where() == null ? subSel.where() : new GridSqlOperation(AND, parent.where(), subSel.where()));
+ GridSqlAst where = subSel.where();
+ if (where != null) {
Review comment:
Empty line before if.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] tledkov-gridgain merged pull request #9441: IGNITE-14358 GridSubqueryJoinOptimizer fixes and tests.
Posted by GitBox <gi...@apache.org>.
tledkov-gridgain merged pull request #9441:
URL: https://github.com/apache/ignite/pull/9441
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] Berkof commented on a change in pull request #9441: IGNITE-14358 GridSubqueryJoinOptimizer fixes and tests.
Posted by GitBox <gi...@apache.org>.
Berkof commented on a change in pull request #9441:
URL: https://github.com/apache/ignite/pull/9441#discussion_r731585216
##########
File path: modules/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/GridSubqueryJoinOptimizer.java
##########
@@ -399,33 +420,95 @@ private static boolean pullOutSubQryFromTableList(
/**
* Remap all columns that satisfy the predicate such they be referred to the given table.
*
- * @param ast Tree where to search columns.
+ * @param parent Tree where to search columns.
+ * @param subSelect Tree where to search column aliases.
* @param colPred Collection predicate.
* @param tbl Table.
*/
- private static void remapColumns(GridSqlAst ast, Predicate<GridSqlColumn> colPred, GridSqlAlias tbl) {
+ private static void remapColumns(GridSqlAst parent, GridSqlAst subSelect, Predicate<GridSqlColumn> colPred, GridSqlAlias tbl) {
ASTNodeFinder colFinder = new ASTNodeFinder(
- ast,
- (p, c) -> c instanceof GridSqlColumn && colPred.test((GridSqlColumn)c)
- );
+ parent,
+ (p, c) -> c instanceof GridSqlColumn && colPred.test((GridSqlColumn)c),
+ ast -> ast != null && !(ast instanceof GridSqlAlias)
+ ); // only columns without aliases
+
+ remapFoundColumn(colFinder, subSelect, tbl, false);
+
+ ASTNodeFinder aliasFinder = new ASTNodeFinder(
+ parent,
+ (p, c) -> c instanceof GridSqlAlias
+ ); // only aliases
+
+ ASTNodeFinder.Result res;
+
+ while ((res = aliasFinder.findNext()) != null) {
+ GridSqlAlias alias = res.getEl().child(res.getIdx());
+ ASTNodeFinder aliasColFinder = new ASTNodeFinder(
+ alias,
+ (p, c) -> c instanceof GridSqlColumn && colPred.test((GridSqlColumn)c)
+ ); //only columns under the alias
+
+ remapFoundColumn(aliasColFinder, subSelect, tbl, true);
+ }
+ }
+
+ /**
+ * Remap all columns of given finder such they be referred to the given table.
+ *
+ * @param colFinder ASTNodeFinder with Column nodes.
+ * @param subSelect Tree where to search column aliases.
+ * @param tbl Table.
+ * @param underAlias Indicates if column is under alias or not.
+ */
+ private static void remapFoundColumn(ASTNodeFinder colFinder, GridSqlAst subSelect, GridSqlAlias tbl, boolean underAlias) {
ASTNodeFinder.Result res;
while ((res = colFinder.findNext()) != null) {
GridSqlColumn oldCol = res.getEl().child(res.getIdx());
- res.getEl().child(
- res.getIdx(),
- new GridSqlColumn(
- oldCol.column(),
- tbl,
- oldCol.schema(),
- tbl.alias(),
- oldCol.columnName()
- )
- );
+ BiPredicate<GridSqlAst, GridSqlAst> constPred = (p, c) ->
+ c != null && c.getSQL().equals(oldCol.columnName());
+
+ BiPredicate<GridSqlAst, GridSqlAst> aliasPred = (p, c) ->
+ c instanceof GridSqlAlias && ((GridSqlAlias)c).alias().equals(oldCol.columnName());
+
+ ASTNodeFinder.Result aliasOrPred = findNode(subSelect, constPred.or(aliasPred));
+
+ if (aliasOrPred != null) {
+ GridSqlAst child = aliasOrPred.getEl().child(aliasOrPred.getIdx());
+
+ if (!(child instanceof GridSqlAlias) || !underAlias)
+ res.getEl().child(res.getIdx(), child);
+ else
+ res.getEl().child(res.getIdx(), child.child());
+ } else {
Review comment:
}
else {
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org