You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2022/08/29 08:52:17 UTC
[GitHub] [doris] qzsee opened a new pull request, #12151: [Nereids][Feature] Constant fold of an expression
qzsee opened a new pull request, #12151:
URL: https://github.com/apache/doris/pull/12151
# Proposed changes
Issue Number: close #xxx
## Problem summary
Describe your changes.
## Checklist(Required)
1. Does it affect the original behavior:
- [ ] Yes
- [ ] No
- [ ] I don't know
2. Has unit tests been added:
- [ ] Yes
- [ ] No
- [ ] No Need
3. Has document been added or modified:
- [ ] Yes
- [ ] No
- [ ] No Need
4. Does it need to update dependencies:
- [ ] Yes
- [ ] No
5. Are there any changes that cannot be rolled back:
- [ ] Yes (If Yes, please explain WHY)
- [ ] No
## Further comments
If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] 924060929 commented on a diff in pull request #12151: [Nereids][Feature] Constant fold of an expression
Posted by GitBox <gi...@apache.org>.
924060929 commented on code in PR #12151:
URL: https://github.com/apache/doris/pull/12151#discussion_r959230265
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rewrite/rules/FoldConstantRule.java:
##########
@@ -0,0 +1,440 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.expression.rewrite.rules;
+
+import org.apache.doris.analysis.BetweenPredicate;
+import org.apache.doris.analysis.CastExpr;
+import org.apache.doris.analysis.Expr;
+import org.apache.doris.analysis.ExprId;
+import org.apache.doris.analysis.LiteralExpr;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.PrimitiveType;
+import org.apache.doris.catalog.Type;
+import org.apache.doris.common.IdGenerator;
+import org.apache.doris.common.LoadException;
+import org.apache.doris.common.util.TimeUtils;
+import org.apache.doris.common.util.VectorizedUtil;
+import org.apache.doris.nereids.glue.translator.ExpressionTranslator;
+import org.apache.doris.nereids.rules.expression.rewrite.AbstractExpressionRewriteRule;
+import org.apache.doris.nereids.rules.expression.rewrite.ExpressionRewriteContext;
+import org.apache.doris.nereids.trees.expressions.And;
+import org.apache.doris.nereids.trees.expressions.BinaryArithmetic;
+import org.apache.doris.nereids.trees.expressions.CaseWhen;
+import org.apache.doris.nereids.trees.expressions.Cast;
+import org.apache.doris.nereids.trees.expressions.ComparisonPredicate;
+import org.apache.doris.nereids.trees.expressions.CompoundPredicate;
+import org.apache.doris.nereids.trees.expressions.EqualTo;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.ExpressionEvaluator;
+import org.apache.doris.nereids.trees.expressions.GreaterThan;
+import org.apache.doris.nereids.trees.expressions.GreaterThanEqual;
+import org.apache.doris.nereids.trees.expressions.InPredicate;
+import org.apache.doris.nereids.trees.expressions.IsNull;
+import org.apache.doris.nereids.trees.expressions.LessThan;
+import org.apache.doris.nereids.trees.expressions.LessThanEqual;
+import org.apache.doris.nereids.trees.expressions.Like;
+import org.apache.doris.nereids.trees.expressions.Not;
+import org.apache.doris.nereids.trees.expressions.NullSafeEqual;
+import org.apache.doris.nereids.trees.expressions.Or;
+import org.apache.doris.nereids.trees.expressions.TimestampArithmetic;
+import org.apache.doris.nereids.trees.expressions.WhenClause;
+import org.apache.doris.nereids.trees.expressions.functions.BoundFunction;
+import org.apache.doris.nereids.trees.expressions.literal.BooleanLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.Literal;
+import org.apache.doris.nereids.trees.expressions.literal.NullLiteral;
+import org.apache.doris.nereids.types.DataType;
+import org.apache.doris.proto.InternalService;
+import org.apache.doris.proto.InternalService.PConstantExprResult;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.rpc.BackendServiceProxy;
+import org.apache.doris.system.Backend;
+import org.apache.doris.thrift.TExpr;
+import org.apache.doris.thrift.TFoldConstantParams;
+import org.apache.doris.thrift.TNetworkAddress;
+import org.apache.doris.thrift.TPrimitiveType;
+import org.apache.doris.thrift.TQueryGlobals;
+
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.text.DateFormat;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Future;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
+
+/**
+ * Constant evaluation of an expression.
+ */
+public class FoldConstantRule extends AbstractExpressionRewriteRule {
+ public static final FoldConstantRule INSTANCE = new FoldConstantRule();
+ private static final Logger LOG = LogManager.getLogger(FoldConstantRule.class);
+ private static final DateFormat DATE_FORMAT = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
+ private final IdGenerator<ExprId> idGenerator = ExprId.createGenerator();
+
+ private static boolean isAllLiteral(Expression... children) {
+ return Arrays.stream(children).allMatch(c -> c instanceof Literal);
+ }
+
+ private static boolean isAllLiteral(List<Expression> children) {
+ return children.stream().allMatch(c -> c instanceof Literal);
+ }
+
+ private static boolean hasNull(List<Expression> children) {
+ return children.stream().anyMatch(c -> c instanceof NullLiteral);
+ }
+
+ private static boolean hasNull(Expression... children) {
+ return Arrays.stream(children).anyMatch(c -> c instanceof NullLiteral);
+ }
+
+ @Override
+ public Expression rewrite(Expression expr, ExpressionRewriteContext ctx) {
+ Expression expression = expr.accept(this, ctx);
+ if (ctx.connectContext != null && ctx.connectContext.getSessionVariable().isEnableFoldConstantByBe()) {
+ return foldByBe(expression, ctx);
+ }
+ return expression;
+ }
+
+ @Override
+ public Expression visitComparisonPredicate(ComparisonPredicate cp, ExpressionRewriteContext context) {
+ Expression left = rewrite(cp.left(), context);
+ Expression right = rewrite(cp.right(), context);
+
+ if (!(cp instanceof NullSafeEqual) && hasNull(left, right)) {
+ return Literal.of(null);
+ }
+
+ if (isAllLiteral(left, right)) {
+ Literal l = (Literal) left;
+ Literal r = (Literal) right;
+ if (cp instanceof EqualTo) {
+ return BooleanLiteral.of(l.compareTo(r) == 0);
+ }
+ if (cp instanceof GreaterThan) {
+ return BooleanLiteral.of(l.compareTo(r) > 0);
+ }
+ if (cp instanceof GreaterThanEqual) {
+ return BooleanLiteral.of(l.compareTo(r) >= 0);
+ }
+ if (cp instanceof LessThan) {
+ return BooleanLiteral.of(l.compareTo(r) < 0);
+ }
+ if (cp instanceof LessThanEqual) {
+ return BooleanLiteral.of(l.compareTo(r) <= 0);
+ }
+ if (cp instanceof NullSafeEqual) {
+ if (l.isNull() && r.isNull()) {
+ return BooleanLiteral.TRUE;
+ } else if (!l.isNull() && !r.isNull()) {
+ return BooleanLiteral.of(l.compareTo(r) == 0);
+ } else {
+ return BooleanLiteral.FALSE;
+ }
+ }
+ }
+ return cp.withChildren(left, right);
+ }
+
+ @Override
+ public Expression visitNot(Not not, ExpressionRewriteContext context) {
+ Expression child = rewrite(not.child(), context);
+ if (child instanceof NullLiteral) {
+ return Literal.of(null);
+ }
Review Comment:
this part look like a common logic, you can move to the common visit method, like this:
```java
public Expression visit(Expression expr, ExpressionRewriteContext context) {
List<Expression> children = expr.children()
.stream()
.map(child -> child.visit(this, context)
.collect(Collectors.toList());
boolean hasNullLiteralChild = ExpresstionUtils.hasNullLiteral(children);
// isDeterminate: Env.getCurrentEnv().isNullResultWithOneNullParamFunction()
if (expr.isDeterminate() && hasNullLiteralChild) {
return Literal.of(null);
}
if (!ExpressionUtils.isAllLiteral(children) {
return expr;
}
// this position every child is Literal, and forward to visitXxx method
return expr.withChildren(children).accept(this, context);
}
```
so every visitXxx method can simple fold constant because every child must be Literal
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] morrySnow commented on a diff in pull request #12151: [Nereids][Feature] Constant fold of an expression
Posted by GitBox <gi...@apache.org>.
morrySnow commented on code in PR #12151:
URL: https://github.com/apache/doris/pull/12151#discussion_r977225255
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rewrite/rules/TypeCoercion.java:
##########
@@ -82,6 +85,23 @@ public Expression visitBinaryOperator(BinaryOperator binaryOperator, ExpressionR
.orElse(binaryOperator.withChildren(left, right));
}
+ @Override
+ public Expression visitDivide(Divide divide, ExpressionRewriteContext context) {
Review Comment:
pls remove TODO at L48
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java:
##########
@@ -945,6 +946,9 @@ private Expression withPredicate(Expression valueExpression, PredicateContext ct
);
}
break;
+ case DorisParser.NULL:
Review Comment:
pls remove TODO at L388 in function `visitPredicated`
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/util/ExpressionUtils.java:
##########
@@ -257,4 +259,20 @@ public static List<Expression> mergeArguments(Object... arguments) {
}
return builder.build();
}
+
+ public static boolean isAllLiteral(Expression... children) {
+ return Arrays.stream(children).allMatch(c -> c instanceof Literal);
+ }
+
+ public static boolean isAllLiteral(List<Expression> children) {
+ return children.stream().allMatch(c -> c instanceof Literal);
+ }
+
+ public static boolean hasNullLiteral(List<Expression> children) {
+ return children.stream().anyMatch(c -> c instanceof NullLiteral);
+ }
+
+ public static boolean allNullLiteral(List<Expression> children) {
Review Comment:
```suggestion
public static boolean isAllNullLiteral(List<Expression> children) {
```
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/ExpressionEvaluator.java:
##########
@@ -0,0 +1,210 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.trees.expressions;
+
+import org.apache.doris.catalog.Env;
+import org.apache.doris.common.AnalysisException;
+import org.apache.doris.nereids.trees.expressions.functions.ExecutableFunctions;
+import org.apache.doris.nereids.trees.expressions.functions.agg.AggregateFunction;
+import org.apache.doris.nereids.trees.expressions.literal.Literal;
+import org.apache.doris.nereids.trees.expressions.literal.NullLiteral;
+import org.apache.doris.nereids.types.DataType;
+import org.apache.doris.rewrite.FEFunction;
+import org.apache.doris.rewrite.FEFunctionList;
+
+import com.google.common.collect.ImmutableMultimap;
+
+import java.lang.reflect.InvocationTargetException;
+import java.lang.reflect.Method;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.List;
+
+/**
+ * An expression evaluator that evaluates the value of an expression.
+ */
+public enum ExpressionEvaluator {
+ INSTANCE;
+ private ImmutableMultimap<String, FunctionInvoker> functions;
+
+ ExpressionEvaluator() {
+ registerFunctions();
+ }
+
+ /**
+ * Evaluate the value of the expression.
+ */
+ public Expression eval(Expression expression) {
+
+ if (!expression.isConstant() || expression instanceof AggregateFunction) {
+ return expression;
+ }
+
+ String fnName = null;
+ DataType[] args = null;
+ if (expression instanceof BinaryArithmetic) {
+ BinaryArithmetic arithmetic = (BinaryArithmetic) expression;
+ fnName = arithmetic.getLegacyOperator().getName();
+ args = new DataType[]{arithmetic.left().getDataType(), arithmetic.right().getDataType()};
+ } else if (expression instanceof TimestampArithmetic) {
+ TimestampArithmetic arithmetic = (TimestampArithmetic) expression;
+ fnName = arithmetic.getFuncName();
+ args = new DataType[]{arithmetic.left().getDataType(), arithmetic.right().getDataType()};
+ }
+
+ if ((Env.getCurrentEnv().isNullResultWithOneNullParamFunction(fnName))) {
+ for (Expression e : expression.children()) {
+ if (e instanceof NullLiteral) {
+ return Literal.of(null);
+ }
+ }
+ }
+
+ return invoke(expression, fnName, args);
+ }
+
+ private Expression invoke(Expression expression, String fnName, DataType[] args) {
+ FunctionSignature signature = new FunctionSignature(fnName, args, null);
+ FunctionInvoker invoker = getFunction(signature);
+ if (invoker != null) {
+ try {
+ return invoker.invoke(expression.children());
+ } catch (AnalysisException e) {
+ return expression;
+ }
+ }
+ return expression;
+ }
+
+ private FunctionInvoker getFunction(FunctionSignature signature) {
+ Collection<FunctionInvoker> functionInvokers = functions.get(signature.getName());
+ if (functionInvokers == null) {
+ return null;
+ }
+ for (FunctionInvoker invoker : functionInvokers) {
+ DataType[] argTypes1 = invoker.getSignature().getArgTypes();
+ DataType[] argTypes2 = signature.getArgTypes();
Review Comment:
```suggestion
DataType[] candidateTypes = invoker.getSignature().getArgTypes();
DataType[] expectedTypes = signature.getArgTypes();
```
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rewrite/rules/FoldConstantRuleOnBE.java:
##########
@@ -0,0 +1,196 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.expression.rewrite.rules;
+
+import org.apache.doris.analysis.Expr;
+import org.apache.doris.analysis.ExprId;
+import org.apache.doris.analysis.LiteralExpr;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.PrimitiveType;
+import org.apache.doris.catalog.Type;
+import org.apache.doris.common.IdGenerator;
+import org.apache.doris.common.LoadException;
+import org.apache.doris.common.util.TimeUtils;
+import org.apache.doris.common.util.VectorizedUtil;
+import org.apache.doris.nereids.glue.translator.ExpressionTranslator;
+import org.apache.doris.nereids.rules.expression.rewrite.AbstractExpressionRewriteRule;
+import org.apache.doris.nereids.rules.expression.rewrite.ExpressionRewriteContext;
+import org.apache.doris.nereids.trees.expressions.Between;
+import org.apache.doris.nereids.trees.expressions.Cast;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.literal.Literal;
+import org.apache.doris.nereids.types.DataType;
+import org.apache.doris.proto.InternalService;
+import org.apache.doris.proto.InternalService.PConstantExprResult;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.rpc.BackendServiceProxy;
+import org.apache.doris.system.Backend;
+import org.apache.doris.thrift.TExpr;
+import org.apache.doris.thrift.TFoldConstantParams;
+import org.apache.doris.thrift.TNetworkAddress;
+import org.apache.doris.thrift.TPrimitiveType;
+import org.apache.doris.thrift.TQueryGlobals;
+
+import com.google.common.collect.Maps;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.time.LocalDateTime;
+import java.time.format.DateTimeFormatter;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Objects;
+import java.util.concurrent.Future;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * Constant evaluation of an expression.
+ */
+public class FoldConstantRuleOnBE extends AbstractExpressionRewriteRule {
+ public static final FoldConstantRuleOnBE INSTANCE = new FoldConstantRuleOnBE();
+ private static final Logger LOG = LogManager.getLogger(FoldConstantRuleOnBE.class);
+ private static final DateTimeFormatter DATE_FORMAT = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
+ private final IdGenerator<ExprId> idGenerator = ExprId.createGenerator();
+
+ @Override
+ public Expression rewrite(Expression expr, ExpressionRewriteContext ctx) {
+ Expression expression = FoldConstantRuleOnFE.INSTANCE.rewrite(expr, ctx);
+ return foldByBe(expression, ctx);
+ }
+
+ private Expression foldByBe(Expression root, ExpressionRewriteContext context) {
+ Map<String, Expression> constMap = Maps.newHashMap();
+ Map<String, TExpr> staleConstTExprMap = Maps.newHashMap();
+ collectConst(root, constMap, staleConstTExprMap);
+ if (constMap.isEmpty()) {
+ return root;
+ }
+ Map<String, Map<String, TExpr>> paramMap = new HashMap<>();
+ paramMap.put("0", staleConstTExprMap);
+ Map<String, Expression> resultMap = evalOnBe(paramMap, constMap, context.connectContext);
+ if (!resultMap.isEmpty()) {
+ return replace(root, constMap, resultMap);
+ }
+ return root;
+ }
+
+ private Expression replace(Expression root, Map<String, Expression> constMap, Map<String, Expression> resultMap) {
+ for (Entry<String, Expression> entry : constMap.entrySet()) {
+ if (entry.getValue().equals(root)) {
+ return resultMap.get(entry.getKey());
+ }
+ }
+ List<Expression> newChildren = new ArrayList<>();
+ boolean hasNewChildren = false;
+ for (Expression child : root.children()) {
+ Expression newChild = replace(child, constMap, resultMap);
+ if (newChild != child) {
+ hasNewChildren = true;
+ }
+ newChildren.add(newChild);
+ }
+ return hasNewChildren ? root.withChildren(newChildren) : root;
+ }
+
+ private void collectConst(Expression expr, Map<String, Expression> constMap, Map<String, TExpr> tExprMap) {
+ if (expr.isConstant()) {
+ if (expr instanceof Cast) {
+ if (((Cast) expr).child().isNullLiteral()) {
+ return;
+ }
+ }
+ if (expr.isLiteral()) {
+ return;
+ }
+ if (expr instanceof Between) {
+ return;
+ }
+ String id = idGenerator.getNextId().toString();
+ constMap.put(id, expr);
+ Expr staleExpr = ExpressionTranslator.translate(expr, null);
+ tExprMap.put(id, staleExpr.treeToThrift());
+ } else {
+ for (int i = 0; i < expr.children().size(); i++) {
+ final Expression child = expr.children().get(i);
+ collectConst(child, constMap, tExprMap);
+ }
+ }
+ }
+
+ private Map<String, Expression> evalOnBe(Map<String, Map<String, TExpr>> paramMap,
+ Map<String, Expression> constMap, ConnectContext context) {
+
+ Map<String, Expression> resultMap = new HashMap<>();
+ try {
+ List<Long> backendIds = Env.getCurrentSystemInfo().getBackendIds(true);
+ if (backendIds.isEmpty()) {
+ throw new LoadException("No alive backends");
Review Comment:
why LoadException
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rewrite/rules/FoldConstantRuleOnFE.java:
##########
@@ -0,0 +1,310 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.expression.rewrite.rules;
+
+import org.apache.doris.nereids.rules.expression.rewrite.AbstractExpressionRewriteRule;
+import org.apache.doris.nereids.rules.expression.rewrite.ExpressionRewriteContext;
+import org.apache.doris.nereids.trees.expressions.And;
+import org.apache.doris.nereids.trees.expressions.BinaryArithmetic;
+import org.apache.doris.nereids.trees.expressions.CaseWhen;
+import org.apache.doris.nereids.trees.expressions.Cast;
+import org.apache.doris.nereids.trees.expressions.EqualTo;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.ExpressionEvaluator;
+import org.apache.doris.nereids.trees.expressions.GreaterThan;
+import org.apache.doris.nereids.trees.expressions.GreaterThanEqual;
+import org.apache.doris.nereids.trees.expressions.InPredicate;
+import org.apache.doris.nereids.trees.expressions.IsNull;
+import org.apache.doris.nereids.trees.expressions.LessThan;
+import org.apache.doris.nereids.trees.expressions.LessThanEqual;
+import org.apache.doris.nereids.trees.expressions.Like;
+import org.apache.doris.nereids.trees.expressions.Not;
+import org.apache.doris.nereids.trees.expressions.NullSafeEqual;
+import org.apache.doris.nereids.trees.expressions.Or;
+import org.apache.doris.nereids.trees.expressions.Slot;
+import org.apache.doris.nereids.trees.expressions.TimestampArithmetic;
+import org.apache.doris.nereids.trees.expressions.WhenClause;
+import org.apache.doris.nereids.trees.expressions.functions.BoundFunction;
+import org.apache.doris.nereids.trees.expressions.functions.PropagateNullable;
+import org.apache.doris.nereids.trees.expressions.literal.BooleanLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.Literal;
+import org.apache.doris.nereids.util.ExpressionUtils;
+
+import com.google.common.collect.Lists;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * evaluate an expression on fe.
+ */
+public class FoldConstantRuleOnFE extends AbstractExpressionRewriteRule {
+ public static final FoldConstantRuleOnFE INSTANCE = new FoldConstantRuleOnFE();
+
+ @Override
+ public Expression rewrite(Expression expr, ExpressionRewriteContext ctx) {
+ return process(expr, ctx);
+ }
+
+ @Override
+ public Expression visit(Expression expr, ExpressionRewriteContext context) {
+ return expr;
+ }
+
+ /**
+ * process constant expression.
+ */
+ public Expression process(Expression expr, ExpressionRewriteContext ctx) {
+ if (expr instanceof PropagateNullable) {
Review Comment:
do we need to fold children first? in other words, do a postorder traversal
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/util/TypeCoercionUtils.java:
##########
@@ -202,6 +207,39 @@ public static Optional<DataType> findTightestCommonType(DataType left, DataType
return Optional.ofNullable(tightestCommonType);
}
+ /**
+ * The type used for arithmetic operations.
+ */
+ public static DataType getNumResultType(DataType type) {
+ if (type.isTinyIntType() || type.isSmallIntType() || type.isIntType() || type.isBigIntType()) {
+ return BigIntType.INSTANCE;
+ } else if (type.isLargeIntType()) {
+ return LargeIntType.INSTANCE;
+ } else if (type.isFloatType() || type.isDoubleType() || type.isStringType()) {
+ return DoubleType.INSTANCE;
+ } else if (type.isDecimalType()) {
+ return DecimalType.SYSTEM_DEFAULT;
+ } else if (type.isNullType()) {
+ return NullType.INSTANCE;
+ }
+ throw new AnalysisException("no found appropriate data type.");
+ }
+
+ /**
+ * The common type used by arithmetic operations.
+ */
+ public static DataType findCommonType(DataType t1, DataType t2) {
Review Comment:
findCommonNumericsType
##########
fe/fe-core/src/test/java/org/apache/doris/nereids/rules/expression/rewrite/ExpressionRewriteTest.java:
##########
@@ -65,7 +65,7 @@ public void testNotRewrite() {
@Test
public void testNormalizeExpressionRewrite() {
- executor = new ExpressionRuleExecutor(NormalizeBinaryPredicatesRule.INSTANCE);
+ executor = new ExpressionRuleExecutor(ImmutableList.of(NormalizeBinaryPredicatesRule.INSTANCE), null);
Review Comment:
nit: add another constructor with one parameter
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/ExecutableFunctions.java:
##########
@@ -0,0 +1,243 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.trees.expressions.functions;
+
+import org.apache.doris.nereids.exceptions.AnalysisException;
+import org.apache.doris.nereids.trees.expressions.literal.BigIntLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.DateTimeLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.DecimalLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.DoubleLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.IntegerLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.SmallIntLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.TinyIntLiteral;
+import org.apache.doris.rewrite.FEFunction;
+
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.math.BigDecimal;
+
+/**
+ * functions that can be executed in FE.
+ */
+public class ExecutableFunctions {
+ public static final ExecutableFunctions INSTANCE = new ExecutableFunctions();
+
+ private static final Logger LOG = LogManager.getLogger(ExecutableFunctions.class);
+
+ /**
+ * Executable arithmetic functions
+ */
+
+ @FEFunction(name = "add", argTypes = {"TINYINT", "TINYINT"}, returnType = "TINYINT")
Review Comment:
The annotion 'FEFunction' is the one the legacy plan used? does it affect the legacy plan const fold?
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rewrite/rules/FoldConstantRuleOnBE.java:
##########
@@ -0,0 +1,196 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.expression.rewrite.rules;
+
+import org.apache.doris.analysis.Expr;
+import org.apache.doris.analysis.ExprId;
+import org.apache.doris.analysis.LiteralExpr;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.PrimitiveType;
+import org.apache.doris.catalog.Type;
+import org.apache.doris.common.IdGenerator;
+import org.apache.doris.common.LoadException;
+import org.apache.doris.common.util.TimeUtils;
+import org.apache.doris.common.util.VectorizedUtil;
+import org.apache.doris.nereids.glue.translator.ExpressionTranslator;
+import org.apache.doris.nereids.rules.expression.rewrite.AbstractExpressionRewriteRule;
+import org.apache.doris.nereids.rules.expression.rewrite.ExpressionRewriteContext;
+import org.apache.doris.nereids.trees.expressions.Between;
+import org.apache.doris.nereids.trees.expressions.Cast;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.literal.Literal;
+import org.apache.doris.nereids.types.DataType;
+import org.apache.doris.proto.InternalService;
+import org.apache.doris.proto.InternalService.PConstantExprResult;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.rpc.BackendServiceProxy;
+import org.apache.doris.system.Backend;
+import org.apache.doris.thrift.TExpr;
+import org.apache.doris.thrift.TFoldConstantParams;
+import org.apache.doris.thrift.TNetworkAddress;
+import org.apache.doris.thrift.TPrimitiveType;
+import org.apache.doris.thrift.TQueryGlobals;
+
+import com.google.common.collect.Maps;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.time.LocalDateTime;
+import java.time.format.DateTimeFormatter;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Objects;
+import java.util.concurrent.Future;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * Constant evaluation of an expression.
+ */
+public class FoldConstantRuleOnBE extends AbstractExpressionRewriteRule {
+ public static final FoldConstantRuleOnBE INSTANCE = new FoldConstantRuleOnBE();
+ private static final Logger LOG = LogManager.getLogger(FoldConstantRuleOnBE.class);
+ private static final DateTimeFormatter DATE_FORMAT = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
+ private final IdGenerator<ExprId> idGenerator = ExprId.createGenerator();
+
+ @Override
+ public Expression rewrite(Expression expr, ExpressionRewriteContext ctx) {
+ Expression expression = FoldConstantRuleOnFE.INSTANCE.rewrite(expr, ctx);
+ return foldByBe(expression, ctx);
+ }
+
+ private Expression foldByBe(Expression root, ExpressionRewriteContext context) {
Review Comment:
```suggestion
private Expression foldByBE(Expression root, ExpressionRewriteContext context) {
```
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rewrite/rules/FoldConstantRuleOnFE.java:
##########
@@ -0,0 +1,310 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.expression.rewrite.rules;
+
+import org.apache.doris.nereids.rules.expression.rewrite.AbstractExpressionRewriteRule;
+import org.apache.doris.nereids.rules.expression.rewrite.ExpressionRewriteContext;
+import org.apache.doris.nereids.trees.expressions.And;
+import org.apache.doris.nereids.trees.expressions.BinaryArithmetic;
+import org.apache.doris.nereids.trees.expressions.CaseWhen;
+import org.apache.doris.nereids.trees.expressions.Cast;
+import org.apache.doris.nereids.trees.expressions.EqualTo;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.ExpressionEvaluator;
+import org.apache.doris.nereids.trees.expressions.GreaterThan;
+import org.apache.doris.nereids.trees.expressions.GreaterThanEqual;
+import org.apache.doris.nereids.trees.expressions.InPredicate;
+import org.apache.doris.nereids.trees.expressions.IsNull;
+import org.apache.doris.nereids.trees.expressions.LessThan;
+import org.apache.doris.nereids.trees.expressions.LessThanEqual;
+import org.apache.doris.nereids.trees.expressions.Like;
+import org.apache.doris.nereids.trees.expressions.Not;
+import org.apache.doris.nereids.trees.expressions.NullSafeEqual;
+import org.apache.doris.nereids.trees.expressions.Or;
+import org.apache.doris.nereids.trees.expressions.Slot;
+import org.apache.doris.nereids.trees.expressions.TimestampArithmetic;
+import org.apache.doris.nereids.trees.expressions.WhenClause;
+import org.apache.doris.nereids.trees.expressions.functions.BoundFunction;
+import org.apache.doris.nereids.trees.expressions.functions.PropagateNullable;
+import org.apache.doris.nereids.trees.expressions.literal.BooleanLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.Literal;
+import org.apache.doris.nereids.util.ExpressionUtils;
+
+import com.google.common.collect.Lists;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * evaluate an expression on fe.
+ */
+public class FoldConstantRuleOnFE extends AbstractExpressionRewriteRule {
+ public static final FoldConstantRuleOnFE INSTANCE = new FoldConstantRuleOnFE();
+
+ @Override
+ public Expression rewrite(Expression expr, ExpressionRewriteContext ctx) {
+ return process(expr, ctx);
+ }
+
+ @Override
+ public Expression visit(Expression expr, ExpressionRewriteContext context) {
+ return expr;
+ }
+
+ /**
+ * process constant expression.
+ */
+ public Expression process(Expression expr, ExpressionRewriteContext ctx) {
+ if (expr instanceof PropagateNullable) {
+ List<Expression> children = expr.children()
+ .stream()
+ .map(child -> process(child, ctx))
+ .collect(Collectors.toList());
+
+ if (ExpressionUtils.hasNullLiteral(children)) {
+ return Literal.of(null);
+ }
+
+ if (!ExpressionUtils.isAllLiteral(children)) {
+ return expr.withChildren(children);
+ }
+ return expr.withChildren(children).accept(this, ctx);
+ } else {
+ return expr.accept(this, ctx);
+ }
+ }
+
+ @Override
+ public Expression visitEqualTo(EqualTo equalTo, ExpressionRewriteContext context) {
+ return BooleanLiteral.of(((Literal) equalTo.left()).compareTo((Literal) equalTo.right()) == 0);
Review Comment:
where to ensure all children are literal?
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/ExpressionEvaluator.java:
##########
@@ -0,0 +1,210 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.trees.expressions;
+
+import org.apache.doris.catalog.Env;
+import org.apache.doris.common.AnalysisException;
+import org.apache.doris.nereids.trees.expressions.functions.ExecutableFunctions;
+import org.apache.doris.nereids.trees.expressions.functions.agg.AggregateFunction;
+import org.apache.doris.nereids.trees.expressions.literal.Literal;
+import org.apache.doris.nereids.trees.expressions.literal.NullLiteral;
+import org.apache.doris.nereids.types.DataType;
+import org.apache.doris.rewrite.FEFunction;
+import org.apache.doris.rewrite.FEFunctionList;
+
+import com.google.common.collect.ImmutableMultimap;
+
+import java.lang.reflect.InvocationTargetException;
+import java.lang.reflect.Method;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.List;
+
+/**
+ * An expression evaluator that evaluates the value of an expression.
+ */
+public enum ExpressionEvaluator {
+ INSTANCE;
+ private ImmutableMultimap<String, FunctionInvoker> functions;
+
+ ExpressionEvaluator() {
+ registerFunctions();
+ }
+
+ /**
+ * Evaluate the value of the expression.
+ */
+ public Expression eval(Expression expression) {
+
+ if (!expression.isConstant() || expression instanceof AggregateFunction) {
+ return expression;
+ }
+
+ String fnName = null;
+ DataType[] args = null;
+ if (expression instanceof BinaryArithmetic) {
+ BinaryArithmetic arithmetic = (BinaryArithmetic) expression;
+ fnName = arithmetic.getLegacyOperator().getName();
+ args = new DataType[]{arithmetic.left().getDataType(), arithmetic.right().getDataType()};
+ } else if (expression instanceof TimestampArithmetic) {
+ TimestampArithmetic arithmetic = (TimestampArithmetic) expression;
+ fnName = arithmetic.getFuncName();
+ args = new DataType[]{arithmetic.left().getDataType(), arithmetic.right().getDataType()};
+ }
+
+ if ((Env.getCurrentEnv().isNullResultWithOneNullParamFunction(fnName))) {
+ for (Expression e : expression.children()) {
+ if (e instanceof NullLiteral) {
+ return Literal.of(null);
+ }
+ }
+ }
+
+ return invoke(expression, fnName, args);
+ }
+
+ private Expression invoke(Expression expression, String fnName, DataType[] args) {
+ FunctionSignature signature = new FunctionSignature(fnName, args, null);
+ FunctionInvoker invoker = getFunction(signature);
+ if (invoker != null) {
+ try {
+ return invoker.invoke(expression.children());
+ } catch (AnalysisException e) {
+ return expression;
+ }
+ }
+ return expression;
+ }
+
+ private FunctionInvoker getFunction(FunctionSignature signature) {
+ Collection<FunctionInvoker> functionInvokers = functions.get(signature.getName());
+ if (functionInvokers == null) {
+ return null;
+ }
+ for (FunctionInvoker invoker : functionInvokers) {
Review Comment:
```suggestion
for (FunctionInvoker candidate : functionInvokers) {
```
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rewrite/rules/FoldConstantRuleOnBE.java:
##########
@@ -0,0 +1,196 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.expression.rewrite.rules;
+
+import org.apache.doris.analysis.Expr;
+import org.apache.doris.analysis.ExprId;
+import org.apache.doris.analysis.LiteralExpr;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.PrimitiveType;
+import org.apache.doris.catalog.Type;
+import org.apache.doris.common.IdGenerator;
+import org.apache.doris.common.LoadException;
+import org.apache.doris.common.util.TimeUtils;
+import org.apache.doris.common.util.VectorizedUtil;
+import org.apache.doris.nereids.glue.translator.ExpressionTranslator;
+import org.apache.doris.nereids.rules.expression.rewrite.AbstractExpressionRewriteRule;
+import org.apache.doris.nereids.rules.expression.rewrite.ExpressionRewriteContext;
+import org.apache.doris.nereids.trees.expressions.Between;
+import org.apache.doris.nereids.trees.expressions.Cast;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.literal.Literal;
+import org.apache.doris.nereids.types.DataType;
+import org.apache.doris.proto.InternalService;
+import org.apache.doris.proto.InternalService.PConstantExprResult;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.rpc.BackendServiceProxy;
+import org.apache.doris.system.Backend;
+import org.apache.doris.thrift.TExpr;
+import org.apache.doris.thrift.TFoldConstantParams;
+import org.apache.doris.thrift.TNetworkAddress;
+import org.apache.doris.thrift.TPrimitiveType;
+import org.apache.doris.thrift.TQueryGlobals;
+
+import com.google.common.collect.Maps;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.time.LocalDateTime;
+import java.time.format.DateTimeFormatter;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Objects;
+import java.util.concurrent.Future;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * Constant evaluation of an expression.
+ */
+public class FoldConstantRuleOnBE extends AbstractExpressionRewriteRule {
+ public static final FoldConstantRuleOnBE INSTANCE = new FoldConstantRuleOnBE();
+ private static final Logger LOG = LogManager.getLogger(FoldConstantRuleOnBE.class);
+ private static final DateTimeFormatter DATE_FORMAT = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
+ private final IdGenerator<ExprId> idGenerator = ExprId.createGenerator();
+
+ @Override
+ public Expression rewrite(Expression expr, ExpressionRewriteContext ctx) {
+ Expression expression = FoldConstantRuleOnFE.INSTANCE.rewrite(expr, ctx);
+ return foldByBe(expression, ctx);
+ }
+
+ private Expression foldByBe(Expression root, ExpressionRewriteContext context) {
+ Map<String, Expression> constMap = Maps.newHashMap();
+ Map<String, TExpr> staleConstTExprMap = Maps.newHashMap();
+ collectConst(root, constMap, staleConstTExprMap);
+ if (constMap.isEmpty()) {
+ return root;
+ }
+ Map<String, Map<String, TExpr>> paramMap = new HashMap<>();
+ paramMap.put("0", staleConstTExprMap);
+ Map<String, Expression> resultMap = evalOnBe(paramMap, constMap, context.connectContext);
+ if (!resultMap.isEmpty()) {
+ return replace(root, constMap, resultMap);
+ }
+ return root;
+ }
+
+ private Expression replace(Expression root, Map<String, Expression> constMap, Map<String, Expression> resultMap) {
+ for (Entry<String, Expression> entry : constMap.entrySet()) {
+ if (entry.getValue().equals(root)) {
+ return resultMap.get(entry.getKey());
+ }
+ }
+ List<Expression> newChildren = new ArrayList<>();
+ boolean hasNewChildren = false;
+ for (Expression child : root.children()) {
+ Expression newChild = replace(child, constMap, resultMap);
+ if (newChild != child) {
+ hasNewChildren = true;
+ }
+ newChildren.add(newChild);
+ }
+ return hasNewChildren ? root.withChildren(newChildren) : root;
+ }
+
+ private void collectConst(Expression expr, Map<String, Expression> constMap, Map<String, TExpr> tExprMap) {
+ if (expr.isConstant()) {
+ if (expr instanceof Cast) {
Review Comment:
nit: add some comment to describe why need to process these expr specially
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] qzsee commented on a diff in pull request #12151: [Nereids][Feature] Constant fold of an expression
Posted by GitBox <gi...@apache.org>.
qzsee commented on code in PR #12151:
URL: https://github.com/apache/doris/pull/12151#discussion_r978232138
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/ExecutableFunctions.java:
##########
@@ -0,0 +1,243 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.trees.expressions.functions;
+
+import org.apache.doris.nereids.exceptions.AnalysisException;
+import org.apache.doris.nereids.trees.expressions.literal.BigIntLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.DateTimeLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.DecimalLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.DoubleLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.IntegerLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.SmallIntLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.TinyIntLiteral;
+import org.apache.doris.rewrite.FEFunction;
+
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.math.BigDecimal;
+
+/**
+ * functions that can be executed in FE.
+ */
+public class ExecutableFunctions {
+ public static final ExecutableFunctions INSTANCE = new ExecutableFunctions();
+
+ private static final Logger LOG = LogManager.getLogger(ExecutableFunctions.class);
+
+ /**
+ * Executable arithmetic functions
+ */
+
+ @FEFunction(name = "add", argTypes = {"TINYINT", "TINYINT"}, returnType = "TINYINT")
Review Comment:
you are right. I fix it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] qzsee commented on a diff in pull request #12151: [Nereids][Feature] Constant fold of an expression
Posted by GitBox <gi...@apache.org>.
qzsee commented on code in PR #12151:
URL: https://github.com/apache/doris/pull/12151#discussion_r963205692
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rewrite/rules/FoldConstantRule.java:
##########
@@ -0,0 +1,440 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.expression.rewrite.rules;
+
+import org.apache.doris.analysis.BetweenPredicate;
+import org.apache.doris.analysis.CastExpr;
+import org.apache.doris.analysis.Expr;
+import org.apache.doris.analysis.ExprId;
+import org.apache.doris.analysis.LiteralExpr;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.PrimitiveType;
+import org.apache.doris.catalog.Type;
+import org.apache.doris.common.IdGenerator;
+import org.apache.doris.common.LoadException;
+import org.apache.doris.common.util.TimeUtils;
+import org.apache.doris.common.util.VectorizedUtil;
+import org.apache.doris.nereids.glue.translator.ExpressionTranslator;
+import org.apache.doris.nereids.rules.expression.rewrite.AbstractExpressionRewriteRule;
+import org.apache.doris.nereids.rules.expression.rewrite.ExpressionRewriteContext;
+import org.apache.doris.nereids.trees.expressions.And;
+import org.apache.doris.nereids.trees.expressions.BinaryArithmetic;
+import org.apache.doris.nereids.trees.expressions.CaseWhen;
+import org.apache.doris.nereids.trees.expressions.Cast;
+import org.apache.doris.nereids.trees.expressions.ComparisonPredicate;
+import org.apache.doris.nereids.trees.expressions.CompoundPredicate;
+import org.apache.doris.nereids.trees.expressions.EqualTo;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.ExpressionEvaluator;
+import org.apache.doris.nereids.trees.expressions.GreaterThan;
+import org.apache.doris.nereids.trees.expressions.GreaterThanEqual;
+import org.apache.doris.nereids.trees.expressions.InPredicate;
+import org.apache.doris.nereids.trees.expressions.IsNull;
+import org.apache.doris.nereids.trees.expressions.LessThan;
+import org.apache.doris.nereids.trees.expressions.LessThanEqual;
+import org.apache.doris.nereids.trees.expressions.Like;
+import org.apache.doris.nereids.trees.expressions.Not;
+import org.apache.doris.nereids.trees.expressions.NullSafeEqual;
+import org.apache.doris.nereids.trees.expressions.Or;
+import org.apache.doris.nereids.trees.expressions.TimestampArithmetic;
+import org.apache.doris.nereids.trees.expressions.WhenClause;
+import org.apache.doris.nereids.trees.expressions.functions.BoundFunction;
+import org.apache.doris.nereids.trees.expressions.literal.BooleanLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.Literal;
+import org.apache.doris.nereids.trees.expressions.literal.NullLiteral;
+import org.apache.doris.nereids.types.DataType;
+import org.apache.doris.proto.InternalService;
+import org.apache.doris.proto.InternalService.PConstantExprResult;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.rpc.BackendServiceProxy;
+import org.apache.doris.system.Backend;
+import org.apache.doris.thrift.TExpr;
+import org.apache.doris.thrift.TFoldConstantParams;
+import org.apache.doris.thrift.TNetworkAddress;
+import org.apache.doris.thrift.TPrimitiveType;
+import org.apache.doris.thrift.TQueryGlobals;
+
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.text.DateFormat;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Future;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
+
+/**
+ * Constant evaluation of an expression.
+ */
+public class FoldConstantRule extends AbstractExpressionRewriteRule {
+ public static final FoldConstantRule INSTANCE = new FoldConstantRule();
+ private static final Logger LOG = LogManager.getLogger(FoldConstantRule.class);
+ private static final DateFormat DATE_FORMAT = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
+ private final IdGenerator<ExprId> idGenerator = ExprId.createGenerator();
+
+ private static boolean isAllLiteral(Expression... children) {
+ return Arrays.stream(children).allMatch(c -> c instanceof Literal);
+ }
+
+ private static boolean isAllLiteral(List<Expression> children) {
+ return children.stream().allMatch(c -> c instanceof Literal);
+ }
+
+ private static boolean hasNull(List<Expression> children) {
+ return children.stream().anyMatch(c -> c instanceof NullLiteral);
+ }
+
+ private static boolean hasNull(Expression... children) {
+ return Arrays.stream(children).anyMatch(c -> c instanceof NullLiteral);
+ }
+
+ @Override
+ public Expression rewrite(Expression expr, ExpressionRewriteContext ctx) {
+ Expression expression = expr.accept(this, ctx);
+ if (ctx.connectContext != null && ctx.connectContext.getSessionVariable().isEnableFoldConstantByBe()) {
+ return foldByBe(expression, ctx);
+ }
+ return expression;
+ }
+
+ @Override
+ public Expression visitComparisonPredicate(ComparisonPredicate cp, ExpressionRewriteContext context) {
+ Expression left = rewrite(cp.left(), context);
+ Expression right = rewrite(cp.right(), context);
+
+ if (!(cp instanceof NullSafeEqual) && hasNull(left, right)) {
+ return Literal.of(null);
+ }
+
+ if (isAllLiteral(left, right)) {
+ Literal l = (Literal) left;
+ Literal r = (Literal) right;
+ if (cp instanceof EqualTo) {
+ return BooleanLiteral.of(l.compareTo(r) == 0);
+ }
+ if (cp instanceof GreaterThan) {
+ return BooleanLiteral.of(l.compareTo(r) > 0);
+ }
+ if (cp instanceof GreaterThanEqual) {
+ return BooleanLiteral.of(l.compareTo(r) >= 0);
+ }
+ if (cp instanceof LessThan) {
+ return BooleanLiteral.of(l.compareTo(r) < 0);
+ }
+ if (cp instanceof LessThanEqual) {
+ return BooleanLiteral.of(l.compareTo(r) <= 0);
+ }
+ if (cp instanceof NullSafeEqual) {
+ if (l.isNull() && r.isNull()) {
+ return BooleanLiteral.TRUE;
+ } else if (!l.isNull() && !r.isNull()) {
+ return BooleanLiteral.of(l.compareTo(r) == 0);
+ } else {
+ return BooleanLiteral.FALSE;
+ }
+ }
+ }
+ return cp.withChildren(left, right);
+ }
+
+ @Override
+ public Expression visitNot(Not not, ExpressionRewriteContext context) {
+ Expression child = rewrite(not.child(), context);
+ if (child instanceof NullLiteral) {
+ return Literal.of(null);
+ }
+ if (child.isLiteral()) {
+ if (child instanceof BooleanLiteral) {
+ BooleanLiteral c = (BooleanLiteral) child;
+ return BooleanLiteral.of(!c.getValue());
+ }
+ }
+ return not.withChildren(child);
+ }
+
+ @Override
+ public Expression visitCompoundPredicate(CompoundPredicate compoundPredicate, ExpressionRewriteContext context) {
+ Expression left = rewrite(compoundPredicate.left(), context);
+ Expression right = rewrite(compoundPredicate.right(), context);
+ if (left instanceof NullLiteral && right instanceof NullLiteral) {
+ return Literal.of(null);
+ }
+ if (left instanceof Literal || right instanceof Literal) {
+ if (compoundPredicate instanceof And) {
+ if (left.equals(BooleanLiteral.FALSE) || right.equals(BooleanLiteral.FALSE)) {
+ return BooleanLiteral.FALSE;
+ }
+ if (left.equals(BooleanLiteral.TRUE)) {
+ return right;
+ }
+ if (right.equals(BooleanLiteral.TRUE)) {
+ return left;
+ }
+ }
+ if (compoundPredicate instanceof Or) {
+ if (left.equals(BooleanLiteral.TRUE) || right.equals(BooleanLiteral.TRUE)) {
+ return BooleanLiteral.TRUE;
+ }
+ if (left.equals(BooleanLiteral.FALSE)) {
+ return right;
+ }
+ if (right.equals(BooleanLiteral.FALSE)) {
+ return left;
+ }
+ }
+ }
+ return compoundPredicate.withChildren(left, right);
+ }
+
+ @Override
+ public Expression visitLike(Like like, ExpressionRewriteContext context) {
+ Expression left = rewrite(like.left(), context);
+ if (left instanceof NullLiteral) {
+ return Literal.of(null);
+ }
+ return like.withChildren(left, like.right());
+ }
+
+ @Override
+ public Expression visitCast(Cast cast, ExpressionRewriteContext context) {
+ if (hasNull(cast.children())) {
+ return Literal.of(null);
+ }
+ Expression child = rewrite(cast.child(), context);
+ if (child.isLiteral()) {
+ return child.castTo(cast.getDataType());
+ }
+ return cast;
+ }
+
+ @Override
+ public Expression visitBoundFunction(BoundFunction boundFunction, ExpressionRewriteContext context) {
+ List<Expression> newArgs = boundFunction.getArguments().stream().map(arg -> rewrite(arg, context))
+ .collect(Collectors.toList());
+ if (isAllLiteral(newArgs)) {
+ return ExpressionEvaluator.INSTANCE.eval(boundFunction.withChildren(newArgs));
+ }
+ return boundFunction.withChildren(newArgs);
+ }
+
+ @Override
+ public Expression visitBinaryArithmetic(BinaryArithmetic binaryArithmetic, ExpressionRewriteContext context) {
+ Expression left = rewrite(binaryArithmetic.left(), context);
+ Expression right = rewrite(binaryArithmetic.right(), context);
+ if (left instanceof NullLiteral || right instanceof NullLiteral) {
+ return Literal.of(null);
+ }
+ if (isAllLiteral(left, right)) {
+ return ExpressionEvaluator.INSTANCE.eval(binaryArithmetic.withChildren(left, right));
+ }
+ return binaryArithmetic.withChildren(left, right);
+ }
+
+ @Override
+ public Expression visitCaseWhen(CaseWhen caseWhen, ExpressionRewriteContext context) {
+ Expression newDefault = null;
+ boolean foundNewDefault = false;
+
+ List<WhenClause> whenClauses = new ArrayList<>();
+ for (WhenClause whenClause : caseWhen.getWhenClauses()) {
+ Expression whenOperand = rewrite(whenClause.getOperand(), context);
+
+ if (!(whenOperand instanceof Literal)) {
+ whenClauses.add(new WhenClause(whenOperand, rewrite(whenClause.getResult(), context)));
+ } else if (BooleanLiteral.TRUE.equals(whenOperand)) {
+ foundNewDefault = true;
+ newDefault = rewrite(whenClause.getResult(), context);
+ break;
+ }
+ }
+
+ Expression defaultResult;
+ if (foundNewDefault) {
+ defaultResult = newDefault;
+ } else {
+ defaultResult = rewrite(caseWhen.getDefaultValue().orElse(Literal.of(null)), context);
+ }
+
+ if (whenClauses.isEmpty()) {
+ return defaultResult;
+ }
+ return new CaseWhen(whenClauses, defaultResult);
+ }
+
+ @Override
+ public Expression visitInPredicate(InPredicate inPredicate, ExpressionRewriteContext context) {
+ List<Expression> newChildren = inPredicate.children().stream().map(c -> rewrite(c, context))
+ .collect(Collectors.toList());
+ if (newChildren.get(0) instanceof NullLiteral) {
+ return Literal.of(null);
+ }
+ if (isAllLiteral(newChildren)) {
+ Literal c0 = (Literal) newChildren.get(0);
+ for (int i = 1; i < newChildren.size(); i++) {
+ if (c0.compareTo((Literal) newChildren.get(i)) == 0) {
+ return Literal.of(true);
+ }
+ }
+ return Literal.of(false);
+ }
+ return inPredicate.withChildren(newChildren);
+ }
+
+ @Override
+ public Expression visitIsNull(IsNull isNull, ExpressionRewriteContext context) {
+ Expression child = rewrite(isNull.child(), context);
+ if (child instanceof NullLiteral) {
+ return Literal.of(true);
+ } else if (child instanceof Literal) {
+ return Literal.of(false);
+ }
+ return isNull.withChildren(child);
+ }
+
+ @Override
+ public Expression visitTimestampArithmetic(TimestampArithmetic arithmetic, ExpressionRewriteContext context) {
+ Expression left = rewrite(arithmetic.child(0), context);
+ Expression right = rewrite(arithmetic.child(1), context);
+ if (isAllLiteral(left, right)) {
+ return ExpressionEvaluator.INSTANCE.eval(arithmetic.withChildren(left, right));
+ }
+ return arithmetic.withChildren(left, right);
+ }
+
+ private Expression foldByBe(Expression root, ExpressionRewriteContext context) {
+ if (root.isConstant() && !root.isLiteral()) {
+ Expr expr = ExpressionTranslator.INSTANCE.translate(root, null);
+ assignId(expr);
+ Map<String, Expr> ori = new HashMap<>();
+ ori.put(expr.getId().toString(), expr);
+
+ Map<String, Map<String, TExpr>> paramMap = new HashMap<>();
+ Map<String, TExpr> constMap = new HashMap<>();
+
+ collectConst(expr, constMap);
+ paramMap.put("0", constMap);
Review Comment:
This is just a token, so it doesn't make sense, but it does get confusing, so use the ID of the current expression instead
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] 924060929 commented on a diff in pull request #12151: [Nereids][Feature] Constant fold of an expression
Posted by GitBox <gi...@apache.org>.
924060929 commented on code in PR #12151:
URL: https://github.com/apache/doris/pull/12151#discussion_r959230265
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rewrite/rules/FoldConstantRule.java:
##########
@@ -0,0 +1,440 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.expression.rewrite.rules;
+
+import org.apache.doris.analysis.BetweenPredicate;
+import org.apache.doris.analysis.CastExpr;
+import org.apache.doris.analysis.Expr;
+import org.apache.doris.analysis.ExprId;
+import org.apache.doris.analysis.LiteralExpr;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.PrimitiveType;
+import org.apache.doris.catalog.Type;
+import org.apache.doris.common.IdGenerator;
+import org.apache.doris.common.LoadException;
+import org.apache.doris.common.util.TimeUtils;
+import org.apache.doris.common.util.VectorizedUtil;
+import org.apache.doris.nereids.glue.translator.ExpressionTranslator;
+import org.apache.doris.nereids.rules.expression.rewrite.AbstractExpressionRewriteRule;
+import org.apache.doris.nereids.rules.expression.rewrite.ExpressionRewriteContext;
+import org.apache.doris.nereids.trees.expressions.And;
+import org.apache.doris.nereids.trees.expressions.BinaryArithmetic;
+import org.apache.doris.nereids.trees.expressions.CaseWhen;
+import org.apache.doris.nereids.trees.expressions.Cast;
+import org.apache.doris.nereids.trees.expressions.ComparisonPredicate;
+import org.apache.doris.nereids.trees.expressions.CompoundPredicate;
+import org.apache.doris.nereids.trees.expressions.EqualTo;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.ExpressionEvaluator;
+import org.apache.doris.nereids.trees.expressions.GreaterThan;
+import org.apache.doris.nereids.trees.expressions.GreaterThanEqual;
+import org.apache.doris.nereids.trees.expressions.InPredicate;
+import org.apache.doris.nereids.trees.expressions.IsNull;
+import org.apache.doris.nereids.trees.expressions.LessThan;
+import org.apache.doris.nereids.trees.expressions.LessThanEqual;
+import org.apache.doris.nereids.trees.expressions.Like;
+import org.apache.doris.nereids.trees.expressions.Not;
+import org.apache.doris.nereids.trees.expressions.NullSafeEqual;
+import org.apache.doris.nereids.trees.expressions.Or;
+import org.apache.doris.nereids.trees.expressions.TimestampArithmetic;
+import org.apache.doris.nereids.trees.expressions.WhenClause;
+import org.apache.doris.nereids.trees.expressions.functions.BoundFunction;
+import org.apache.doris.nereids.trees.expressions.literal.BooleanLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.Literal;
+import org.apache.doris.nereids.trees.expressions.literal.NullLiteral;
+import org.apache.doris.nereids.types.DataType;
+import org.apache.doris.proto.InternalService;
+import org.apache.doris.proto.InternalService.PConstantExprResult;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.rpc.BackendServiceProxy;
+import org.apache.doris.system.Backend;
+import org.apache.doris.thrift.TExpr;
+import org.apache.doris.thrift.TFoldConstantParams;
+import org.apache.doris.thrift.TNetworkAddress;
+import org.apache.doris.thrift.TPrimitiveType;
+import org.apache.doris.thrift.TQueryGlobals;
+
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.text.DateFormat;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Future;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
+
+/**
+ * Constant evaluation of an expression.
+ */
+public class FoldConstantRule extends AbstractExpressionRewriteRule {
+ public static final FoldConstantRule INSTANCE = new FoldConstantRule();
+ private static final Logger LOG = LogManager.getLogger(FoldConstantRule.class);
+ private static final DateFormat DATE_FORMAT = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
+ private final IdGenerator<ExprId> idGenerator = ExprId.createGenerator();
+
+ private static boolean isAllLiteral(Expression... children) {
+ return Arrays.stream(children).allMatch(c -> c instanceof Literal);
+ }
+
+ private static boolean isAllLiteral(List<Expression> children) {
+ return children.stream().allMatch(c -> c instanceof Literal);
+ }
+
+ private static boolean hasNull(List<Expression> children) {
+ return children.stream().anyMatch(c -> c instanceof NullLiteral);
+ }
+
+ private static boolean hasNull(Expression... children) {
+ return Arrays.stream(children).anyMatch(c -> c instanceof NullLiteral);
+ }
+
+ @Override
+ public Expression rewrite(Expression expr, ExpressionRewriteContext ctx) {
+ Expression expression = expr.accept(this, ctx);
+ if (ctx.connectContext != null && ctx.connectContext.getSessionVariable().isEnableFoldConstantByBe()) {
+ return foldByBe(expression, ctx);
+ }
+ return expression;
+ }
+
+ @Override
+ public Expression visitComparisonPredicate(ComparisonPredicate cp, ExpressionRewriteContext context) {
+ Expression left = rewrite(cp.left(), context);
+ Expression right = rewrite(cp.right(), context);
+
+ if (!(cp instanceof NullSafeEqual) && hasNull(left, right)) {
+ return Literal.of(null);
+ }
+
+ if (isAllLiteral(left, right)) {
+ Literal l = (Literal) left;
+ Literal r = (Literal) right;
+ if (cp instanceof EqualTo) {
+ return BooleanLiteral.of(l.compareTo(r) == 0);
+ }
+ if (cp instanceof GreaterThan) {
+ return BooleanLiteral.of(l.compareTo(r) > 0);
+ }
+ if (cp instanceof GreaterThanEqual) {
+ return BooleanLiteral.of(l.compareTo(r) >= 0);
+ }
+ if (cp instanceof LessThan) {
+ return BooleanLiteral.of(l.compareTo(r) < 0);
+ }
+ if (cp instanceof LessThanEqual) {
+ return BooleanLiteral.of(l.compareTo(r) <= 0);
+ }
+ if (cp instanceof NullSafeEqual) {
+ if (l.isNull() && r.isNull()) {
+ return BooleanLiteral.TRUE;
+ } else if (!l.isNull() && !r.isNull()) {
+ return BooleanLiteral.of(l.compareTo(r) == 0);
+ } else {
+ return BooleanLiteral.FALSE;
+ }
+ }
+ }
+ return cp.withChildren(left, right);
+ }
+
+ @Override
+ public Expression visitNot(Not not, ExpressionRewriteContext context) {
+ Expression child = rewrite(not.child(), context);
+ if (child instanceof NullLiteral) {
+ return Literal.of(null);
+ }
Review Comment:
this part look like a common logic, you can move to the common visit method, like this:
```java
public Expression visit(Expression expr, ExpressionRewriteContext context) {
List<Expression> children = expr.children()
.stream()
.map(child -> child.visit(this, context)
.collect(Collectors.toList());
boolean hasNullLiteralChild = ExpresstionUtils.hasNullLiteral(children);
// isNullHandling: Env.getCurrentEnv().isNullResultWithOneNullParamFunction()
if (expr.isNullHandling() && hasNullLiteralChild) {
return Literal.of(null);
}
if (!ExpressionUtils.isAllLiteral(children) {
return expr.withChildren(children);
}
// this position every child is Literal, and forward to visitXxx method
return expr.withChildren(children).accept(this, context);
}
```
so every visitXxx method can simple fold constant because every child must be Literal
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] 924060929 commented on a diff in pull request #12151: [Nereids][Feature] Constant fold of an expression
Posted by GitBox <gi...@apache.org>.
924060929 commented on code in PR #12151:
URL: https://github.com/apache/doris/pull/12151#discussion_r959212369
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/BindFunction.java:
##########
@@ -143,24 +141,14 @@ public Expression visitUnboundFunction(UnboundFunction unboundFunction, Void con
@Override
public Expression visitTimestampArithmetic(TimestampArithmetic arithmetic, Void context) {
- String funcOpName = null;
+ String funcOpName;
if (arithmetic.getFuncName() == null) {
funcOpName = String.format("%sS_%s", arithmetic.getTimeUnit(),
Review Comment:
this place should add some example comment, e.g.
```java
// e.g. YEARS_ADD, MONTHS_SUB
```
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/glue/translator/ExpressionTranslator.java:
##########
@@ -226,8 +226,14 @@ public Expr visitCaseWhen(CaseWhen caseWhen, PlanTranslatorContext context) {
@Override
public Expr visitCast(Cast cast, PlanTranslatorContext context) {
// left child of cast is expression, right child of cast is target type
- return new CastExpr(cast.getDataType().toCatalogDataType(),
- cast.child().accept(this, context));
+ Expr expr = cast.child().accept(this, context);
+ try {
+ expr.finalizeForNereids();
Review Comment:
why finalize at this position? why the unified finalize in ExpressionTranslator.translate can not process this case?
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rewrite/rules/FoldConstantRule.java:
##########
@@ -0,0 +1,440 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.expression.rewrite.rules;
+
+import org.apache.doris.analysis.BetweenPredicate;
+import org.apache.doris.analysis.CastExpr;
+import org.apache.doris.analysis.Expr;
+import org.apache.doris.analysis.ExprId;
+import org.apache.doris.analysis.LiteralExpr;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.PrimitiveType;
+import org.apache.doris.catalog.Type;
+import org.apache.doris.common.IdGenerator;
+import org.apache.doris.common.LoadException;
+import org.apache.doris.common.util.TimeUtils;
+import org.apache.doris.common.util.VectorizedUtil;
+import org.apache.doris.nereids.glue.translator.ExpressionTranslator;
+import org.apache.doris.nereids.rules.expression.rewrite.AbstractExpressionRewriteRule;
+import org.apache.doris.nereids.rules.expression.rewrite.ExpressionRewriteContext;
+import org.apache.doris.nereids.trees.expressions.And;
+import org.apache.doris.nereids.trees.expressions.BinaryArithmetic;
+import org.apache.doris.nereids.trees.expressions.CaseWhen;
+import org.apache.doris.nereids.trees.expressions.Cast;
+import org.apache.doris.nereids.trees.expressions.ComparisonPredicate;
+import org.apache.doris.nereids.trees.expressions.CompoundPredicate;
+import org.apache.doris.nereids.trees.expressions.EqualTo;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.ExpressionEvaluator;
+import org.apache.doris.nereids.trees.expressions.GreaterThan;
+import org.apache.doris.nereids.trees.expressions.GreaterThanEqual;
+import org.apache.doris.nereids.trees.expressions.InPredicate;
+import org.apache.doris.nereids.trees.expressions.IsNull;
+import org.apache.doris.nereids.trees.expressions.LessThan;
+import org.apache.doris.nereids.trees.expressions.LessThanEqual;
+import org.apache.doris.nereids.trees.expressions.Like;
+import org.apache.doris.nereids.trees.expressions.Not;
+import org.apache.doris.nereids.trees.expressions.NullSafeEqual;
+import org.apache.doris.nereids.trees.expressions.Or;
+import org.apache.doris.nereids.trees.expressions.TimestampArithmetic;
+import org.apache.doris.nereids.trees.expressions.WhenClause;
+import org.apache.doris.nereids.trees.expressions.functions.BoundFunction;
+import org.apache.doris.nereids.trees.expressions.literal.BooleanLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.Literal;
+import org.apache.doris.nereids.trees.expressions.literal.NullLiteral;
+import org.apache.doris.nereids.types.DataType;
+import org.apache.doris.proto.InternalService;
+import org.apache.doris.proto.InternalService.PConstantExprResult;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.rpc.BackendServiceProxy;
+import org.apache.doris.system.Backend;
+import org.apache.doris.thrift.TExpr;
+import org.apache.doris.thrift.TFoldConstantParams;
+import org.apache.doris.thrift.TNetworkAddress;
+import org.apache.doris.thrift.TPrimitiveType;
+import org.apache.doris.thrift.TQueryGlobals;
+
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.text.DateFormat;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Future;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
+
+/**
+ * Constant evaluation of an expression.
+ */
+public class FoldConstantRule extends AbstractExpressionRewriteRule {
+ public static final FoldConstantRule INSTANCE = new FoldConstantRule();
+ private static final Logger LOG = LogManager.getLogger(FoldConstantRule.class);
+ private static final DateFormat DATE_FORMAT = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
+ private final IdGenerator<ExprId> idGenerator = ExprId.createGenerator();
+
+ private static boolean isAllLiteral(Expression... children) {
+ return Arrays.stream(children).allMatch(c -> c instanceof Literal);
+ }
+
+ private static boolean isAllLiteral(List<Expression> children) {
+ return children.stream().allMatch(c -> c instanceof Literal);
+ }
+
+ private static boolean hasNull(List<Expression> children) {
+ return children.stream().anyMatch(c -> c instanceof NullLiteral);
+ }
+
+ private static boolean hasNull(Expression... children) {
Review Comment:
```suggestion
private static boolean hasNullLiteral(Expression... children) {
```
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rewrite/rules/FoldConstantRule.java:
##########
@@ -0,0 +1,440 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.expression.rewrite.rules;
+
+import org.apache.doris.analysis.BetweenPredicate;
+import org.apache.doris.analysis.CastExpr;
+import org.apache.doris.analysis.Expr;
+import org.apache.doris.analysis.ExprId;
+import org.apache.doris.analysis.LiteralExpr;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.PrimitiveType;
+import org.apache.doris.catalog.Type;
+import org.apache.doris.common.IdGenerator;
+import org.apache.doris.common.LoadException;
+import org.apache.doris.common.util.TimeUtils;
+import org.apache.doris.common.util.VectorizedUtil;
+import org.apache.doris.nereids.glue.translator.ExpressionTranslator;
+import org.apache.doris.nereids.rules.expression.rewrite.AbstractExpressionRewriteRule;
+import org.apache.doris.nereids.rules.expression.rewrite.ExpressionRewriteContext;
+import org.apache.doris.nereids.trees.expressions.And;
+import org.apache.doris.nereids.trees.expressions.BinaryArithmetic;
+import org.apache.doris.nereids.trees.expressions.CaseWhen;
+import org.apache.doris.nereids.trees.expressions.Cast;
+import org.apache.doris.nereids.trees.expressions.ComparisonPredicate;
+import org.apache.doris.nereids.trees.expressions.CompoundPredicate;
+import org.apache.doris.nereids.trees.expressions.EqualTo;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.ExpressionEvaluator;
+import org.apache.doris.nereids.trees.expressions.GreaterThan;
+import org.apache.doris.nereids.trees.expressions.GreaterThanEqual;
+import org.apache.doris.nereids.trees.expressions.InPredicate;
+import org.apache.doris.nereids.trees.expressions.IsNull;
+import org.apache.doris.nereids.trees.expressions.LessThan;
+import org.apache.doris.nereids.trees.expressions.LessThanEqual;
+import org.apache.doris.nereids.trees.expressions.Like;
+import org.apache.doris.nereids.trees.expressions.Not;
+import org.apache.doris.nereids.trees.expressions.NullSafeEqual;
+import org.apache.doris.nereids.trees.expressions.Or;
+import org.apache.doris.nereids.trees.expressions.TimestampArithmetic;
+import org.apache.doris.nereids.trees.expressions.WhenClause;
+import org.apache.doris.nereids.trees.expressions.functions.BoundFunction;
+import org.apache.doris.nereids.trees.expressions.literal.BooleanLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.Literal;
+import org.apache.doris.nereids.trees.expressions.literal.NullLiteral;
+import org.apache.doris.nereids.types.DataType;
+import org.apache.doris.proto.InternalService;
+import org.apache.doris.proto.InternalService.PConstantExprResult;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.rpc.BackendServiceProxy;
+import org.apache.doris.system.Backend;
+import org.apache.doris.thrift.TExpr;
+import org.apache.doris.thrift.TFoldConstantParams;
+import org.apache.doris.thrift.TNetworkAddress;
+import org.apache.doris.thrift.TPrimitiveType;
+import org.apache.doris.thrift.TQueryGlobals;
+
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.text.DateFormat;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Future;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
+
+/**
+ * Constant evaluation of an expression.
+ */
+public class FoldConstantRule extends AbstractExpressionRewriteRule {
+ public static final FoldConstantRule INSTANCE = new FoldConstantRule();
+ private static final Logger LOG = LogManager.getLogger(FoldConstantRule.class);
+ private static final DateFormat DATE_FORMAT = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
+ private final IdGenerator<ExprId> idGenerator = ExprId.createGenerator();
+
+ private static boolean isAllLiteral(Expression... children) {
+ return Arrays.stream(children).allMatch(c -> c instanceof Literal);
+ }
+
+ private static boolean isAllLiteral(List<Expression> children) {
+ return children.stream().allMatch(c -> c instanceof Literal);
+ }
+
+ private static boolean hasNull(List<Expression> children) {
Review Comment:
```suggestion
private static boolean hasNullLiteral(List<Expression> children) {
```
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java:
##########
@@ -916,6 +917,9 @@ private Expression withPredicate(Expression valueExpression, PredicateContext ct
);
}
break;
+ case DorisParser.NULL:
+ outExpression = new IsNull(valueExpression);
Review Comment:
missing Not(IsNull)?
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rewrite/rules/FoldConstantRule.java:
##########
@@ -0,0 +1,440 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.expression.rewrite.rules;
+
+import org.apache.doris.analysis.BetweenPredicate;
+import org.apache.doris.analysis.CastExpr;
+import org.apache.doris.analysis.Expr;
+import org.apache.doris.analysis.ExprId;
+import org.apache.doris.analysis.LiteralExpr;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.PrimitiveType;
+import org.apache.doris.catalog.Type;
+import org.apache.doris.common.IdGenerator;
+import org.apache.doris.common.LoadException;
+import org.apache.doris.common.util.TimeUtils;
+import org.apache.doris.common.util.VectorizedUtil;
+import org.apache.doris.nereids.glue.translator.ExpressionTranslator;
+import org.apache.doris.nereids.rules.expression.rewrite.AbstractExpressionRewriteRule;
+import org.apache.doris.nereids.rules.expression.rewrite.ExpressionRewriteContext;
+import org.apache.doris.nereids.trees.expressions.And;
+import org.apache.doris.nereids.trees.expressions.BinaryArithmetic;
+import org.apache.doris.nereids.trees.expressions.CaseWhen;
+import org.apache.doris.nereids.trees.expressions.Cast;
+import org.apache.doris.nereids.trees.expressions.ComparisonPredicate;
+import org.apache.doris.nereids.trees.expressions.CompoundPredicate;
+import org.apache.doris.nereids.trees.expressions.EqualTo;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.ExpressionEvaluator;
+import org.apache.doris.nereids.trees.expressions.GreaterThan;
+import org.apache.doris.nereids.trees.expressions.GreaterThanEqual;
+import org.apache.doris.nereids.trees.expressions.InPredicate;
+import org.apache.doris.nereids.trees.expressions.IsNull;
+import org.apache.doris.nereids.trees.expressions.LessThan;
+import org.apache.doris.nereids.trees.expressions.LessThanEqual;
+import org.apache.doris.nereids.trees.expressions.Like;
+import org.apache.doris.nereids.trees.expressions.Not;
+import org.apache.doris.nereids.trees.expressions.NullSafeEqual;
+import org.apache.doris.nereids.trees.expressions.Or;
+import org.apache.doris.nereids.trees.expressions.TimestampArithmetic;
+import org.apache.doris.nereids.trees.expressions.WhenClause;
+import org.apache.doris.nereids.trees.expressions.functions.BoundFunction;
+import org.apache.doris.nereids.trees.expressions.literal.BooleanLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.Literal;
+import org.apache.doris.nereids.trees.expressions.literal.NullLiteral;
+import org.apache.doris.nereids.types.DataType;
+import org.apache.doris.proto.InternalService;
+import org.apache.doris.proto.InternalService.PConstantExprResult;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.rpc.BackendServiceProxy;
+import org.apache.doris.system.Backend;
+import org.apache.doris.thrift.TExpr;
+import org.apache.doris.thrift.TFoldConstantParams;
+import org.apache.doris.thrift.TNetworkAddress;
+import org.apache.doris.thrift.TPrimitiveType;
+import org.apache.doris.thrift.TQueryGlobals;
+
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.text.DateFormat;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Future;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
+
+/**
+ * Constant evaluation of an expression.
+ */
+public class FoldConstantRule extends AbstractExpressionRewriteRule {
+ public static final FoldConstantRule INSTANCE = new FoldConstantRule();
+ private static final Logger LOG = LogManager.getLogger(FoldConstantRule.class);
+ private static final DateFormat DATE_FORMAT = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
+ private final IdGenerator<ExprId> idGenerator = ExprId.createGenerator();
+
+ private static boolean isAllLiteral(Expression... children) {
+ return Arrays.stream(children).allMatch(c -> c instanceof Literal);
+ }
+
+ private static boolean isAllLiteral(List<Expression> children) {
+ return children.stream().allMatch(c -> c instanceof Literal);
+ }
+
+ private static boolean hasNull(List<Expression> children) {
+ return children.stream().anyMatch(c -> c instanceof NullLiteral);
+ }
+
+ private static boolean hasNull(Expression... children) {
+ return Arrays.stream(children).anyMatch(c -> c instanceof NullLiteral);
+ }
+
+ @Override
+ public Expression rewrite(Expression expr, ExpressionRewriteContext ctx) {
+ Expression expression = expr.accept(this, ctx);
+ if (ctx.connectContext != null && ctx.connectContext.getSessionVariable().isEnableFoldConstantByBe()) {
+ return foldByBe(expression, ctx);
+ }
+ return expression;
+ }
+
+ @Override
+ public Expression visitComparisonPredicate(ComparisonPredicate cp, ExpressionRewriteContext context) {
+ Expression left = rewrite(cp.left(), context);
+ Expression right = rewrite(cp.right(), context);
+
+ if (!(cp instanceof NullSafeEqual) && hasNull(left, right)) {
+ return Literal.of(null);
+ }
+
+ if (isAllLiteral(left, right)) {
+ Literal l = (Literal) left;
+ Literal r = (Literal) right;
+ if (cp instanceof EqualTo) {
+ return BooleanLiteral.of(l.compareTo(r) == 0);
+ }
+ if (cp instanceof GreaterThan) {
+ return BooleanLiteral.of(l.compareTo(r) > 0);
+ }
+ if (cp instanceof GreaterThanEqual) {
+ return BooleanLiteral.of(l.compareTo(r) >= 0);
+ }
+ if (cp instanceof LessThan) {
+ return BooleanLiteral.of(l.compareTo(r) < 0);
+ }
+ if (cp instanceof LessThanEqual) {
+ return BooleanLiteral.of(l.compareTo(r) <= 0);
+ }
+ if (cp instanceof NullSafeEqual) {
+ if (l.isNull() && r.isNull()) {
+ return BooleanLiteral.TRUE;
+ } else if (!l.isNull() && !r.isNull()) {
+ return BooleanLiteral.of(l.compareTo(r) == 0);
+ } else {
+ return BooleanLiteral.FALSE;
+ }
+ }
+ }
+ return cp.withChildren(left, right);
+ }
+
+ @Override
+ public Expression visitNot(Not not, ExpressionRewriteContext context) {
+ Expression child = rewrite(not.child(), context);
+ if (child instanceof NullLiteral) {
+ return Literal.of(null);
+ }
+ if (child.isLiteral()) {
+ if (child instanceof BooleanLiteral) {
+ BooleanLiteral c = (BooleanLiteral) child;
+ return BooleanLiteral.of(!c.getValue());
+ }
+ }
+ return not.withChildren(child);
+ }
+
+ @Override
+ public Expression visitCompoundPredicate(CompoundPredicate compoundPredicate, ExpressionRewriteContext context) {
+ Expression left = rewrite(compoundPredicate.left(), context);
+ Expression right = rewrite(compoundPredicate.right(), context);
+ if (left instanceof NullLiteral && right instanceof NullLiteral) {
+ return Literal.of(null);
+ }
+ if (left instanceof Literal || right instanceof Literal) {
+ if (compoundPredicate instanceof And) {
+ if (left.equals(BooleanLiteral.FALSE) || right.equals(BooleanLiteral.FALSE)) {
+ return BooleanLiteral.FALSE;
+ }
+ if (left.equals(BooleanLiteral.TRUE)) {
+ return right;
+ }
+ if (right.equals(BooleanLiteral.TRUE)) {
+ return left;
+ }
+ }
+ if (compoundPredicate instanceof Or) {
+ if (left.equals(BooleanLiteral.TRUE) || right.equals(BooleanLiteral.TRUE)) {
+ return BooleanLiteral.TRUE;
+ }
+ if (left.equals(BooleanLiteral.FALSE)) {
+ return right;
+ }
+ if (right.equals(BooleanLiteral.FALSE)) {
+ return left;
+ }
+ }
+ }
+ return compoundPredicate.withChildren(left, right);
+ }
+
+ @Override
+ public Expression visitLike(Like like, ExpressionRewriteContext context) {
+ Expression left = rewrite(like.left(), context);
+ if (left instanceof NullLiteral) {
+ return Literal.of(null);
+ }
+ return like.withChildren(left, like.right());
+ }
+
+ @Override
+ public Expression visitCast(Cast cast, ExpressionRewriteContext context) {
+ if (hasNull(cast.children())) {
+ return Literal.of(null);
+ }
+ Expression child = rewrite(cast.child(), context);
+ if (child.isLiteral()) {
+ return child.castTo(cast.getDataType());
+ }
+ return cast;
+ }
+
+ @Override
+ public Expression visitBoundFunction(BoundFunction boundFunction, ExpressionRewriteContext context) {
+ List<Expression> newArgs = boundFunction.getArguments().stream().map(arg -> rewrite(arg, context))
+ .collect(Collectors.toList());
+ if (isAllLiteral(newArgs)) {
+ return ExpressionEvaluator.INSTANCE.eval(boundFunction.withChildren(newArgs));
+ }
+ return boundFunction.withChildren(newArgs);
+ }
+
+ @Override
+ public Expression visitBinaryArithmetic(BinaryArithmetic binaryArithmetic, ExpressionRewriteContext context) {
+ Expression left = rewrite(binaryArithmetic.left(), context);
+ Expression right = rewrite(binaryArithmetic.right(), context);
+ if (left instanceof NullLiteral || right instanceof NullLiteral) {
+ return Literal.of(null);
+ }
+ if (isAllLiteral(left, right)) {
+ return ExpressionEvaluator.INSTANCE.eval(binaryArithmetic.withChildren(left, right));
+ }
+ return binaryArithmetic.withChildren(left, right);
+ }
+
+ @Override
+ public Expression visitCaseWhen(CaseWhen caseWhen, ExpressionRewriteContext context) {
+ Expression newDefault = null;
+ boolean foundNewDefault = false;
+
+ List<WhenClause> whenClauses = new ArrayList<>();
+ for (WhenClause whenClause : caseWhen.getWhenClauses()) {
+ Expression whenOperand = rewrite(whenClause.getOperand(), context);
+
+ if (!(whenOperand instanceof Literal)) {
+ whenClauses.add(new WhenClause(whenOperand, rewrite(whenClause.getResult(), context)));
+ } else if (BooleanLiteral.TRUE.equals(whenOperand)) {
+ foundNewDefault = true;
+ newDefault = rewrite(whenClause.getResult(), context);
+ break;
+ }
+ }
+
+ Expression defaultResult;
+ if (foundNewDefault) {
+ defaultResult = newDefault;
+ } else {
+ defaultResult = rewrite(caseWhen.getDefaultValue().orElse(Literal.of(null)), context);
+ }
+
+ if (whenClauses.isEmpty()) {
+ return defaultResult;
+ }
+ return new CaseWhen(whenClauses, defaultResult);
+ }
+
+ @Override
+ public Expression visitInPredicate(InPredicate inPredicate, ExpressionRewriteContext context) {
+ List<Expression> newChildren = inPredicate.children().stream().map(c -> rewrite(c, context))
+ .collect(Collectors.toList());
+ if (newChildren.get(0) instanceof NullLiteral) {
+ return Literal.of(null);
+ }
+ if (isAllLiteral(newChildren)) {
+ Literal c0 = (Literal) newChildren.get(0);
+ for (int i = 1; i < newChildren.size(); i++) {
+ if (c0.compareTo((Literal) newChildren.get(i)) == 0) {
+ return Literal.of(true);
+ }
+ }
+ return Literal.of(false);
+ }
+ return inPredicate.withChildren(newChildren);
+ }
+
+ @Override
+ public Expression visitIsNull(IsNull isNull, ExpressionRewriteContext context) {
+ Expression child = rewrite(isNull.child(), context);
+ if (child instanceof NullLiteral) {
+ return Literal.of(true);
+ } else if (child instanceof Literal) {
+ return Literal.of(false);
+ }
+ return isNull.withChildren(child);
+ }
+
+ @Override
+ public Expression visitTimestampArithmetic(TimestampArithmetic arithmetic, ExpressionRewriteContext context) {
+ Expression left = rewrite(arithmetic.child(0), context);
+ Expression right = rewrite(arithmetic.child(1), context);
+ if (isAllLiteral(left, right)) {
+ return ExpressionEvaluator.INSTANCE.eval(arithmetic.withChildren(left, right));
+ }
+ return arithmetic.withChildren(left, right);
+ }
+
+ private Expression foldByBe(Expression root, ExpressionRewriteContext context) {
+ if (root.isConstant() && !root.isLiteral()) {
+ Expr expr = ExpressionTranslator.INSTANCE.translate(root, null);
+ assignId(expr);
+ Map<String, Expr> ori = new HashMap<>();
Review Comment:
what is the ori mean?
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rewrite/rules/FoldConstantRule.java:
##########
@@ -0,0 +1,440 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.expression.rewrite.rules;
+
+import org.apache.doris.analysis.BetweenPredicate;
+import org.apache.doris.analysis.CastExpr;
+import org.apache.doris.analysis.Expr;
+import org.apache.doris.analysis.ExprId;
+import org.apache.doris.analysis.LiteralExpr;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.PrimitiveType;
+import org.apache.doris.catalog.Type;
+import org.apache.doris.common.IdGenerator;
+import org.apache.doris.common.LoadException;
+import org.apache.doris.common.util.TimeUtils;
+import org.apache.doris.common.util.VectorizedUtil;
+import org.apache.doris.nereids.glue.translator.ExpressionTranslator;
+import org.apache.doris.nereids.rules.expression.rewrite.AbstractExpressionRewriteRule;
+import org.apache.doris.nereids.rules.expression.rewrite.ExpressionRewriteContext;
+import org.apache.doris.nereids.trees.expressions.And;
+import org.apache.doris.nereids.trees.expressions.BinaryArithmetic;
+import org.apache.doris.nereids.trees.expressions.CaseWhen;
+import org.apache.doris.nereids.trees.expressions.Cast;
+import org.apache.doris.nereids.trees.expressions.ComparisonPredicate;
+import org.apache.doris.nereids.trees.expressions.CompoundPredicate;
+import org.apache.doris.nereids.trees.expressions.EqualTo;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.ExpressionEvaluator;
+import org.apache.doris.nereids.trees.expressions.GreaterThan;
+import org.apache.doris.nereids.trees.expressions.GreaterThanEqual;
+import org.apache.doris.nereids.trees.expressions.InPredicate;
+import org.apache.doris.nereids.trees.expressions.IsNull;
+import org.apache.doris.nereids.trees.expressions.LessThan;
+import org.apache.doris.nereids.trees.expressions.LessThanEqual;
+import org.apache.doris.nereids.trees.expressions.Like;
+import org.apache.doris.nereids.trees.expressions.Not;
+import org.apache.doris.nereids.trees.expressions.NullSafeEqual;
+import org.apache.doris.nereids.trees.expressions.Or;
+import org.apache.doris.nereids.trees.expressions.TimestampArithmetic;
+import org.apache.doris.nereids.trees.expressions.WhenClause;
+import org.apache.doris.nereids.trees.expressions.functions.BoundFunction;
+import org.apache.doris.nereids.trees.expressions.literal.BooleanLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.Literal;
+import org.apache.doris.nereids.trees.expressions.literal.NullLiteral;
+import org.apache.doris.nereids.types.DataType;
+import org.apache.doris.proto.InternalService;
+import org.apache.doris.proto.InternalService.PConstantExprResult;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.rpc.BackendServiceProxy;
+import org.apache.doris.system.Backend;
+import org.apache.doris.thrift.TExpr;
+import org.apache.doris.thrift.TFoldConstantParams;
+import org.apache.doris.thrift.TNetworkAddress;
+import org.apache.doris.thrift.TPrimitiveType;
+import org.apache.doris.thrift.TQueryGlobals;
+
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.text.DateFormat;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Future;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
+
+/**
+ * Constant evaluation of an expression.
+ */
+public class FoldConstantRule extends AbstractExpressionRewriteRule {
+ public static final FoldConstantRule INSTANCE = new FoldConstantRule();
+ private static final Logger LOG = LogManager.getLogger(FoldConstantRule.class);
+ private static final DateFormat DATE_FORMAT = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
+ private final IdGenerator<ExprId> idGenerator = ExprId.createGenerator();
Review Comment:
singleton object with SimpleDateFormat field is not thread-safe, you should use apache commons DateFormatUtils or DateUtils.
And ExprId seems like non thread-safe, too.
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rewrite/rules/FoldConstantRule.java:
##########
@@ -0,0 +1,440 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.expression.rewrite.rules;
+
+import org.apache.doris.analysis.BetweenPredicate;
+import org.apache.doris.analysis.CastExpr;
+import org.apache.doris.analysis.Expr;
+import org.apache.doris.analysis.ExprId;
+import org.apache.doris.analysis.LiteralExpr;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.PrimitiveType;
+import org.apache.doris.catalog.Type;
+import org.apache.doris.common.IdGenerator;
+import org.apache.doris.common.LoadException;
+import org.apache.doris.common.util.TimeUtils;
+import org.apache.doris.common.util.VectorizedUtil;
+import org.apache.doris.nereids.glue.translator.ExpressionTranslator;
+import org.apache.doris.nereids.rules.expression.rewrite.AbstractExpressionRewriteRule;
+import org.apache.doris.nereids.rules.expression.rewrite.ExpressionRewriteContext;
+import org.apache.doris.nereids.trees.expressions.And;
+import org.apache.doris.nereids.trees.expressions.BinaryArithmetic;
+import org.apache.doris.nereids.trees.expressions.CaseWhen;
+import org.apache.doris.nereids.trees.expressions.Cast;
+import org.apache.doris.nereids.trees.expressions.ComparisonPredicate;
+import org.apache.doris.nereids.trees.expressions.CompoundPredicate;
+import org.apache.doris.nereids.trees.expressions.EqualTo;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.ExpressionEvaluator;
+import org.apache.doris.nereids.trees.expressions.GreaterThan;
+import org.apache.doris.nereids.trees.expressions.GreaterThanEqual;
+import org.apache.doris.nereids.trees.expressions.InPredicate;
+import org.apache.doris.nereids.trees.expressions.IsNull;
+import org.apache.doris.nereids.trees.expressions.LessThan;
+import org.apache.doris.nereids.trees.expressions.LessThanEqual;
+import org.apache.doris.nereids.trees.expressions.Like;
+import org.apache.doris.nereids.trees.expressions.Not;
+import org.apache.doris.nereids.trees.expressions.NullSafeEqual;
+import org.apache.doris.nereids.trees.expressions.Or;
+import org.apache.doris.nereids.trees.expressions.TimestampArithmetic;
+import org.apache.doris.nereids.trees.expressions.WhenClause;
+import org.apache.doris.nereids.trees.expressions.functions.BoundFunction;
+import org.apache.doris.nereids.trees.expressions.literal.BooleanLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.Literal;
+import org.apache.doris.nereids.trees.expressions.literal.NullLiteral;
+import org.apache.doris.nereids.types.DataType;
+import org.apache.doris.proto.InternalService;
+import org.apache.doris.proto.InternalService.PConstantExprResult;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.rpc.BackendServiceProxy;
+import org.apache.doris.system.Backend;
+import org.apache.doris.thrift.TExpr;
+import org.apache.doris.thrift.TFoldConstantParams;
+import org.apache.doris.thrift.TNetworkAddress;
+import org.apache.doris.thrift.TPrimitiveType;
+import org.apache.doris.thrift.TQueryGlobals;
+
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.text.DateFormat;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Future;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
+
+/**
+ * Constant evaluation of an expression.
+ */
+public class FoldConstantRule extends AbstractExpressionRewriteRule {
+ public static final FoldConstantRule INSTANCE = new FoldConstantRule();
+ private static final Logger LOG = LogManager.getLogger(FoldConstantRule.class);
+ private static final DateFormat DATE_FORMAT = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
+ private final IdGenerator<ExprId> idGenerator = ExprId.createGenerator();
+
+ private static boolean isAllLiteral(Expression... children) {
+ return Arrays.stream(children).allMatch(c -> c instanceof Literal);
+ }
+
+ private static boolean isAllLiteral(List<Expression> children) {
+ return children.stream().allMatch(c -> c instanceof Literal);
+ }
+
+ private static boolean hasNull(List<Expression> children) {
+ return children.stream().anyMatch(c -> c instanceof NullLiteral);
+ }
+
+ private static boolean hasNull(Expression... children) {
+ return Arrays.stream(children).anyMatch(c -> c instanceof NullLiteral);
+ }
+
+ @Override
+ public Expression rewrite(Expression expr, ExpressionRewriteContext ctx) {
+ Expression expression = expr.accept(this, ctx);
+ if (ctx.connectContext != null && ctx.connectContext.getSessionVariable().isEnableFoldConstantByBe()) {
+ return foldByBe(expression, ctx);
+ }
+ return expression;
+ }
+
+ @Override
+ public Expression visitComparisonPredicate(ComparisonPredicate cp, ExpressionRewriteContext context) {
+ Expression left = rewrite(cp.left(), context);
+ Expression right = rewrite(cp.right(), context);
+
+ if (!(cp instanceof NullSafeEqual) && hasNull(left, right)) {
+ return Literal.of(null);
+ }
+
+ if (isAllLiteral(left, right)) {
+ Literal l = (Literal) left;
+ Literal r = (Literal) right;
+ if (cp instanceof EqualTo) {
+ return BooleanLiteral.of(l.compareTo(r) == 0);
+ }
+ if (cp instanceof GreaterThan) {
+ return BooleanLiteral.of(l.compareTo(r) > 0);
+ }
+ if (cp instanceof GreaterThanEqual) {
+ return BooleanLiteral.of(l.compareTo(r) >= 0);
+ }
+ if (cp instanceof LessThan) {
+ return BooleanLiteral.of(l.compareTo(r) < 0);
+ }
+ if (cp instanceof LessThanEqual) {
+ return BooleanLiteral.of(l.compareTo(r) <= 0);
+ }
+ if (cp instanceof NullSafeEqual) {
+ if (l.isNull() && r.isNull()) {
+ return BooleanLiteral.TRUE;
+ } else if (!l.isNull() && !r.isNull()) {
+ return BooleanLiteral.of(l.compareTo(r) == 0);
+ } else {
+ return BooleanLiteral.FALSE;
+ }
+ }
Review Comment:
why not overwrite visitEqualTo, visitGreaterThan and so on methods?
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rewrite/rules/FoldConstantRule.java:
##########
@@ -0,0 +1,440 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.expression.rewrite.rules;
+
+import org.apache.doris.analysis.BetweenPredicate;
+import org.apache.doris.analysis.CastExpr;
+import org.apache.doris.analysis.Expr;
+import org.apache.doris.analysis.ExprId;
+import org.apache.doris.analysis.LiteralExpr;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.PrimitiveType;
+import org.apache.doris.catalog.Type;
+import org.apache.doris.common.IdGenerator;
+import org.apache.doris.common.LoadException;
+import org.apache.doris.common.util.TimeUtils;
+import org.apache.doris.common.util.VectorizedUtil;
+import org.apache.doris.nereids.glue.translator.ExpressionTranslator;
+import org.apache.doris.nereids.rules.expression.rewrite.AbstractExpressionRewriteRule;
+import org.apache.doris.nereids.rules.expression.rewrite.ExpressionRewriteContext;
+import org.apache.doris.nereids.trees.expressions.And;
+import org.apache.doris.nereids.trees.expressions.BinaryArithmetic;
+import org.apache.doris.nereids.trees.expressions.CaseWhen;
+import org.apache.doris.nereids.trees.expressions.Cast;
+import org.apache.doris.nereids.trees.expressions.ComparisonPredicate;
+import org.apache.doris.nereids.trees.expressions.CompoundPredicate;
+import org.apache.doris.nereids.trees.expressions.EqualTo;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.ExpressionEvaluator;
+import org.apache.doris.nereids.trees.expressions.GreaterThan;
+import org.apache.doris.nereids.trees.expressions.GreaterThanEqual;
+import org.apache.doris.nereids.trees.expressions.InPredicate;
+import org.apache.doris.nereids.trees.expressions.IsNull;
+import org.apache.doris.nereids.trees.expressions.LessThan;
+import org.apache.doris.nereids.trees.expressions.LessThanEqual;
+import org.apache.doris.nereids.trees.expressions.Like;
+import org.apache.doris.nereids.trees.expressions.Not;
+import org.apache.doris.nereids.trees.expressions.NullSafeEqual;
+import org.apache.doris.nereids.trees.expressions.Or;
+import org.apache.doris.nereids.trees.expressions.TimestampArithmetic;
+import org.apache.doris.nereids.trees.expressions.WhenClause;
+import org.apache.doris.nereids.trees.expressions.functions.BoundFunction;
+import org.apache.doris.nereids.trees.expressions.literal.BooleanLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.Literal;
+import org.apache.doris.nereids.trees.expressions.literal.NullLiteral;
+import org.apache.doris.nereids.types.DataType;
+import org.apache.doris.proto.InternalService;
+import org.apache.doris.proto.InternalService.PConstantExprResult;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.rpc.BackendServiceProxy;
+import org.apache.doris.system.Backend;
+import org.apache.doris.thrift.TExpr;
+import org.apache.doris.thrift.TFoldConstantParams;
+import org.apache.doris.thrift.TNetworkAddress;
+import org.apache.doris.thrift.TPrimitiveType;
+import org.apache.doris.thrift.TQueryGlobals;
+
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.text.DateFormat;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Future;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
+
+/**
+ * Constant evaluation of an expression.
+ */
+public class FoldConstantRule extends AbstractExpressionRewriteRule {
+ public static final FoldConstantRule INSTANCE = new FoldConstantRule();
+ private static final Logger LOG = LogManager.getLogger(FoldConstantRule.class);
+ private static final DateFormat DATE_FORMAT = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
+ private final IdGenerator<ExprId> idGenerator = ExprId.createGenerator();
+
+ private static boolean isAllLiteral(Expression... children) {
+ return Arrays.stream(children).allMatch(c -> c instanceof Literal);
+ }
+
+ private static boolean isAllLiteral(List<Expression> children) {
+ return children.stream().allMatch(c -> c instanceof Literal);
+ }
+
+ private static boolean hasNull(List<Expression> children) {
+ return children.stream().anyMatch(c -> c instanceof NullLiteral);
+ }
+
+ private static boolean hasNull(Expression... children) {
+ return Arrays.stream(children).anyMatch(c -> c instanceof NullLiteral);
+ }
+
+ @Override
+ public Expression rewrite(Expression expr, ExpressionRewriteContext ctx) {
+ Expression expression = expr.accept(this, ctx);
+ if (ctx.connectContext != null && ctx.connectContext.getSessionVariable().isEnableFoldConstantByBe()) {
+ return foldByBe(expression, ctx);
+ }
+ return expression;
+ }
+
+ @Override
+ public Expression visitComparisonPredicate(ComparisonPredicate cp, ExpressionRewriteContext context) {
+ Expression left = rewrite(cp.left(), context);
+ Expression right = rewrite(cp.right(), context);
+
+ if (!(cp instanceof NullSafeEqual) && hasNull(left, right)) {
+ return Literal.of(null);
+ }
+
+ if (isAllLiteral(left, right)) {
+ Literal l = (Literal) left;
+ Literal r = (Literal) right;
+ if (cp instanceof EqualTo) {
+ return BooleanLiteral.of(l.compareTo(r) == 0);
+ }
+ if (cp instanceof GreaterThan) {
+ return BooleanLiteral.of(l.compareTo(r) > 0);
+ }
+ if (cp instanceof GreaterThanEqual) {
+ return BooleanLiteral.of(l.compareTo(r) >= 0);
+ }
+ if (cp instanceof LessThan) {
+ return BooleanLiteral.of(l.compareTo(r) < 0);
+ }
+ if (cp instanceof LessThanEqual) {
+ return BooleanLiteral.of(l.compareTo(r) <= 0);
+ }
+ if (cp instanceof NullSafeEqual) {
+ if (l.isNull() && r.isNull()) {
+ return BooleanLiteral.TRUE;
+ } else if (!l.isNull() && !r.isNull()) {
+ return BooleanLiteral.of(l.compareTo(r) == 0);
+ } else {
+ return BooleanLiteral.FALSE;
+ }
+ }
+ }
+ return cp.withChildren(left, right);
+ }
+
+ @Override
+ public Expression visitNot(Not not, ExpressionRewriteContext context) {
+ Expression child = rewrite(not.child(), context);
+ if (child instanceof NullLiteral) {
+ return Literal.of(null);
+ }
+ if (child.isLiteral()) {
+ if (child instanceof BooleanLiteral) {
+ BooleanLiteral c = (BooleanLiteral) child;
+ return BooleanLiteral.of(!c.getValue());
+ }
+ }
+ return not.withChildren(child);
+ }
+
+ @Override
+ public Expression visitCompoundPredicate(CompoundPredicate compoundPredicate, ExpressionRewriteContext context) {
+ Expression left = rewrite(compoundPredicate.left(), context);
+ Expression right = rewrite(compoundPredicate.right(), context);
+ if (left instanceof NullLiteral && right instanceof NullLiteral) {
+ return Literal.of(null);
+ }
+ if (left instanceof Literal || right instanceof Literal) {
+ if (compoundPredicate instanceof And) {
+ if (left.equals(BooleanLiteral.FALSE) || right.equals(BooleanLiteral.FALSE)) {
+ return BooleanLiteral.FALSE;
+ }
+ if (left.equals(BooleanLiteral.TRUE)) {
+ return right;
+ }
+ if (right.equals(BooleanLiteral.TRUE)) {
+ return left;
+ }
+ }
+ if (compoundPredicate instanceof Or) {
+ if (left.equals(BooleanLiteral.TRUE) || right.equals(BooleanLiteral.TRUE)) {
+ return BooleanLiteral.TRUE;
+ }
+ if (left.equals(BooleanLiteral.FALSE)) {
+ return right;
+ }
+ if (right.equals(BooleanLiteral.FALSE)) {
+ return left;
+ }
+ }
+ }
+ return compoundPredicate.withChildren(left, right);
+ }
+
+ @Override
+ public Expression visitLike(Like like, ExpressionRewriteContext context) {
+ Expression left = rewrite(like.left(), context);
+ if (left instanceof NullLiteral) {
+ return Literal.of(null);
+ }
+ return like.withChildren(left, like.right());
+ }
+
+ @Override
+ public Expression visitCast(Cast cast, ExpressionRewriteContext context) {
+ if (hasNull(cast.children())) {
+ return Literal.of(null);
+ }
+ Expression child = rewrite(cast.child(), context);
+ if (child.isLiteral()) {
+ return child.castTo(cast.getDataType());
+ }
+ return cast;
+ }
+
+ @Override
+ public Expression visitBoundFunction(BoundFunction boundFunction, ExpressionRewriteContext context) {
+ List<Expression> newArgs = boundFunction.getArguments().stream().map(arg -> rewrite(arg, context))
+ .collect(Collectors.toList());
+ if (isAllLiteral(newArgs)) {
+ return ExpressionEvaluator.INSTANCE.eval(boundFunction.withChildren(newArgs));
+ }
+ return boundFunction.withChildren(newArgs);
+ }
+
+ @Override
+ public Expression visitBinaryArithmetic(BinaryArithmetic binaryArithmetic, ExpressionRewriteContext context) {
+ Expression left = rewrite(binaryArithmetic.left(), context);
+ Expression right = rewrite(binaryArithmetic.right(), context);
+ if (left instanceof NullLiteral || right instanceof NullLiteral) {
+ return Literal.of(null);
+ }
+ if (isAllLiteral(left, right)) {
+ return ExpressionEvaluator.INSTANCE.eval(binaryArithmetic.withChildren(left, right));
+ }
+ return binaryArithmetic.withChildren(left, right);
+ }
+
+ @Override
+ public Expression visitCaseWhen(CaseWhen caseWhen, ExpressionRewriteContext context) {
+ Expression newDefault = null;
+ boolean foundNewDefault = false;
+
+ List<WhenClause> whenClauses = new ArrayList<>();
+ for (WhenClause whenClause : caseWhen.getWhenClauses()) {
+ Expression whenOperand = rewrite(whenClause.getOperand(), context);
+
+ if (!(whenOperand instanceof Literal)) {
+ whenClauses.add(new WhenClause(whenOperand, rewrite(whenClause.getResult(), context)));
+ } else if (BooleanLiteral.TRUE.equals(whenOperand)) {
+ foundNewDefault = true;
+ newDefault = rewrite(whenClause.getResult(), context);
+ break;
+ }
+ }
+
+ Expression defaultResult;
+ if (foundNewDefault) {
+ defaultResult = newDefault;
+ } else {
+ defaultResult = rewrite(caseWhen.getDefaultValue().orElse(Literal.of(null)), context);
+ }
+
+ if (whenClauses.isEmpty()) {
+ return defaultResult;
+ }
+ return new CaseWhen(whenClauses, defaultResult);
+ }
+
+ @Override
+ public Expression visitInPredicate(InPredicate inPredicate, ExpressionRewriteContext context) {
+ List<Expression> newChildren = inPredicate.children().stream().map(c -> rewrite(c, context))
+ .collect(Collectors.toList());
+ if (newChildren.get(0) instanceof NullLiteral) {
+ return Literal.of(null);
+ }
+ if (isAllLiteral(newChildren)) {
+ Literal c0 = (Literal) newChildren.get(0);
+ for (int i = 1; i < newChildren.size(); i++) {
+ if (c0.compareTo((Literal) newChildren.get(i)) == 0) {
+ return Literal.of(true);
+ }
+ }
+ return Literal.of(false);
+ }
+ return inPredicate.withChildren(newChildren);
+ }
+
+ @Override
+ public Expression visitIsNull(IsNull isNull, ExpressionRewriteContext context) {
+ Expression child = rewrite(isNull.child(), context);
+ if (child instanceof NullLiteral) {
+ return Literal.of(true);
+ } else if (child instanceof Literal) {
+ return Literal.of(false);
+ }
+ return isNull.withChildren(child);
+ }
+
+ @Override
+ public Expression visitTimestampArithmetic(TimestampArithmetic arithmetic, ExpressionRewriteContext context) {
+ Expression left = rewrite(arithmetic.child(0), context);
+ Expression right = rewrite(arithmetic.child(1), context);
+ if (isAllLiteral(left, right)) {
+ return ExpressionEvaluator.INSTANCE.eval(arithmetic.withChildren(left, right));
+ }
+ return arithmetic.withChildren(left, right);
+ }
+
+ private Expression foldByBe(Expression root, ExpressionRewriteContext context) {
+ if (root.isConstant() && !root.isLiteral()) {
+ Expr expr = ExpressionTranslator.INSTANCE.translate(root, null);
+ assignId(expr);
+ Map<String, Expr> ori = new HashMap<>();
+ ori.put(expr.getId().toString(), expr);
+
+ Map<String, Map<String, TExpr>> paramMap = new HashMap<>();
+ Map<String, TExpr> constMap = new HashMap<>();
+
+ collectConst(expr, constMap);
+ paramMap.put("0", constMap);
+ Map<String, Map<String, Expr>> resMap = calc(paramMap, ori, context.connectContext);
+ Expr result = resMap.get("0").get(expr.getId().toString());
+ if (result instanceof LiteralExpr) {
+ DataType type = DataType.convertFromString(result.getType().getPrimitiveType().toString());
+ return Literal.of(result.getStringValue()).castTo(type);
+ }
+ }
+ return root;
+ }
+
+ private void assignId(Expr expr) {
+ if (expr.getId() == null) {
+ expr.setId(idGenerator.getNextId());
+ }
+ for (Expr child : expr.getChildren()) {
+ assignId(child);
+ }
+ }
+
+ private void collectConst(Expr expr, Map<String, TExpr> constExprMap) {
+ if (expr.isConstant()) {
+ if (expr instanceof CastExpr) {
+ CastExpr castExpr = (CastExpr) expr;
+ if (castExpr.getChild(0) instanceof org.apache.doris.analysis.NullLiteral) {
+ return;
+ }
+ }
+ if (expr instanceof LiteralExpr) {
+ return;
+ }
+ if (expr instanceof BetweenPredicate) {
+ return;
+ }
+ constExprMap.put(expr.getId().toString(), expr.treeToThrift());
+ }
+ for (int i = 0; i < expr.getChildren().size(); i++) {
+ final Expr child = expr.getChildren().get(i);
+ collectConst(child, constExprMap);
+ }
+ }
+
+ private Map<String, Map<String, Expr>> calc(Map<String, Map<String, TExpr>> map, Map<String, Expr> allConstMap,
Review Comment:
rename to meaningful name like executeExpressionInBackend, and rename the parameter names
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rewrite/rules/FoldConstantRule.java:
##########
@@ -0,0 +1,440 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.expression.rewrite.rules;
+
+import org.apache.doris.analysis.BetweenPredicate;
+import org.apache.doris.analysis.CastExpr;
+import org.apache.doris.analysis.Expr;
+import org.apache.doris.analysis.ExprId;
+import org.apache.doris.analysis.LiteralExpr;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.PrimitiveType;
+import org.apache.doris.catalog.Type;
+import org.apache.doris.common.IdGenerator;
+import org.apache.doris.common.LoadException;
+import org.apache.doris.common.util.TimeUtils;
+import org.apache.doris.common.util.VectorizedUtil;
+import org.apache.doris.nereids.glue.translator.ExpressionTranslator;
+import org.apache.doris.nereids.rules.expression.rewrite.AbstractExpressionRewriteRule;
+import org.apache.doris.nereids.rules.expression.rewrite.ExpressionRewriteContext;
+import org.apache.doris.nereids.trees.expressions.And;
+import org.apache.doris.nereids.trees.expressions.BinaryArithmetic;
+import org.apache.doris.nereids.trees.expressions.CaseWhen;
+import org.apache.doris.nereids.trees.expressions.Cast;
+import org.apache.doris.nereids.trees.expressions.ComparisonPredicate;
+import org.apache.doris.nereids.trees.expressions.CompoundPredicate;
+import org.apache.doris.nereids.trees.expressions.EqualTo;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.ExpressionEvaluator;
+import org.apache.doris.nereids.trees.expressions.GreaterThan;
+import org.apache.doris.nereids.trees.expressions.GreaterThanEqual;
+import org.apache.doris.nereids.trees.expressions.InPredicate;
+import org.apache.doris.nereids.trees.expressions.IsNull;
+import org.apache.doris.nereids.trees.expressions.LessThan;
+import org.apache.doris.nereids.trees.expressions.LessThanEqual;
+import org.apache.doris.nereids.trees.expressions.Like;
+import org.apache.doris.nereids.trees.expressions.Not;
+import org.apache.doris.nereids.trees.expressions.NullSafeEqual;
+import org.apache.doris.nereids.trees.expressions.Or;
+import org.apache.doris.nereids.trees.expressions.TimestampArithmetic;
+import org.apache.doris.nereids.trees.expressions.WhenClause;
+import org.apache.doris.nereids.trees.expressions.functions.BoundFunction;
+import org.apache.doris.nereids.trees.expressions.literal.BooleanLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.Literal;
+import org.apache.doris.nereids.trees.expressions.literal.NullLiteral;
+import org.apache.doris.nereids.types.DataType;
+import org.apache.doris.proto.InternalService;
+import org.apache.doris.proto.InternalService.PConstantExprResult;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.rpc.BackendServiceProxy;
+import org.apache.doris.system.Backend;
+import org.apache.doris.thrift.TExpr;
+import org.apache.doris.thrift.TFoldConstantParams;
+import org.apache.doris.thrift.TNetworkAddress;
+import org.apache.doris.thrift.TPrimitiveType;
+import org.apache.doris.thrift.TQueryGlobals;
+
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.text.DateFormat;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Future;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
+
+/**
+ * Constant evaluation of an expression.
+ */
+public class FoldConstantRule extends AbstractExpressionRewriteRule {
+ public static final FoldConstantRule INSTANCE = new FoldConstantRule();
+ private static final Logger LOG = LogManager.getLogger(FoldConstantRule.class);
+ private static final DateFormat DATE_FORMAT = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
+ private final IdGenerator<ExprId> idGenerator = ExprId.createGenerator();
+
+ private static boolean isAllLiteral(Expression... children) {
+ return Arrays.stream(children).allMatch(c -> c instanceof Literal);
+ }
+
+ private static boolean isAllLiteral(List<Expression> children) {
+ return children.stream().allMatch(c -> c instanceof Literal);
+ }
+
+ private static boolean hasNull(List<Expression> children) {
+ return children.stream().anyMatch(c -> c instanceof NullLiteral);
+ }
+
+ private static boolean hasNull(Expression... children) {
+ return Arrays.stream(children).anyMatch(c -> c instanceof NullLiteral);
+ }
+
+ @Override
+ public Expression rewrite(Expression expr, ExpressionRewriteContext ctx) {
+ Expression expression = expr.accept(this, ctx);
+ if (ctx.connectContext != null && ctx.connectContext.getSessionVariable().isEnableFoldConstantByBe()) {
+ return foldByBe(expression, ctx);
+ }
+ return expression;
+ }
+
+ @Override
+ public Expression visitComparisonPredicate(ComparisonPredicate cp, ExpressionRewriteContext context) {
+ Expression left = rewrite(cp.left(), context);
+ Expression right = rewrite(cp.right(), context);
+
+ if (!(cp instanceof NullSafeEqual) && hasNull(left, right)) {
+ return Literal.of(null);
+ }
+
+ if (isAllLiteral(left, right)) {
+ Literal l = (Literal) left;
+ Literal r = (Literal) right;
+ if (cp instanceof EqualTo) {
+ return BooleanLiteral.of(l.compareTo(r) == 0);
+ }
+ if (cp instanceof GreaterThan) {
+ return BooleanLiteral.of(l.compareTo(r) > 0);
+ }
+ if (cp instanceof GreaterThanEqual) {
+ return BooleanLiteral.of(l.compareTo(r) >= 0);
+ }
+ if (cp instanceof LessThan) {
+ return BooleanLiteral.of(l.compareTo(r) < 0);
+ }
+ if (cp instanceof LessThanEqual) {
+ return BooleanLiteral.of(l.compareTo(r) <= 0);
+ }
+ if (cp instanceof NullSafeEqual) {
+ if (l.isNull() && r.isNull()) {
+ return BooleanLiteral.TRUE;
+ } else if (!l.isNull() && !r.isNull()) {
+ return BooleanLiteral.of(l.compareTo(r) == 0);
+ } else {
+ return BooleanLiteral.FALSE;
+ }
+ }
+ }
+ return cp.withChildren(left, right);
+ }
+
+ @Override
+ public Expression visitNot(Not not, ExpressionRewriteContext context) {
+ Expression child = rewrite(not.child(), context);
+ if (child instanceof NullLiteral) {
+ return Literal.of(null);
+ }
Review Comment:
this part look like a common logic, you can move to the common visit method, like this:
```java
public Expression visit(Expression expr, ExpressionRewriteContext context) {
List<Expression> children = expr.children()
.stream()
.map(child -> child.visit(this, context)
.collect(Collectors.toList());
boolean hasNullLiteralChild = ExpresstionUtils.hasNullLiteral(children);
// isDeterminate: Env.getCurrentEnv().isNullResultWithOneNullParamFunction()
if (expr.isDeterminate() && hasNullLiteralChild) {
return expr;
}
if (!ExpressionUtils.isAllLiteral(children) {
return expr;
}
// this position every child is Literal, and forward to visitXxx method
return expr.withChildren(children).accept(this, context);
}
```
so every visitXxx method can simple fold constant because every child must be Literal
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rewrite/rules/FoldConstantRule.java:
##########
@@ -0,0 +1,440 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.expression.rewrite.rules;
+
+import org.apache.doris.analysis.BetweenPredicate;
+import org.apache.doris.analysis.CastExpr;
+import org.apache.doris.analysis.Expr;
+import org.apache.doris.analysis.ExprId;
+import org.apache.doris.analysis.LiteralExpr;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.PrimitiveType;
+import org.apache.doris.catalog.Type;
+import org.apache.doris.common.IdGenerator;
+import org.apache.doris.common.LoadException;
+import org.apache.doris.common.util.TimeUtils;
+import org.apache.doris.common.util.VectorizedUtil;
+import org.apache.doris.nereids.glue.translator.ExpressionTranslator;
+import org.apache.doris.nereids.rules.expression.rewrite.AbstractExpressionRewriteRule;
+import org.apache.doris.nereids.rules.expression.rewrite.ExpressionRewriteContext;
+import org.apache.doris.nereids.trees.expressions.And;
+import org.apache.doris.nereids.trees.expressions.BinaryArithmetic;
+import org.apache.doris.nereids.trees.expressions.CaseWhen;
+import org.apache.doris.nereids.trees.expressions.Cast;
+import org.apache.doris.nereids.trees.expressions.ComparisonPredicate;
+import org.apache.doris.nereids.trees.expressions.CompoundPredicate;
+import org.apache.doris.nereids.trees.expressions.EqualTo;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.ExpressionEvaluator;
+import org.apache.doris.nereids.trees.expressions.GreaterThan;
+import org.apache.doris.nereids.trees.expressions.GreaterThanEqual;
+import org.apache.doris.nereids.trees.expressions.InPredicate;
+import org.apache.doris.nereids.trees.expressions.IsNull;
+import org.apache.doris.nereids.trees.expressions.LessThan;
+import org.apache.doris.nereids.trees.expressions.LessThanEqual;
+import org.apache.doris.nereids.trees.expressions.Like;
+import org.apache.doris.nereids.trees.expressions.Not;
+import org.apache.doris.nereids.trees.expressions.NullSafeEqual;
+import org.apache.doris.nereids.trees.expressions.Or;
+import org.apache.doris.nereids.trees.expressions.TimestampArithmetic;
+import org.apache.doris.nereids.trees.expressions.WhenClause;
+import org.apache.doris.nereids.trees.expressions.functions.BoundFunction;
+import org.apache.doris.nereids.trees.expressions.literal.BooleanLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.Literal;
+import org.apache.doris.nereids.trees.expressions.literal.NullLiteral;
+import org.apache.doris.nereids.types.DataType;
+import org.apache.doris.proto.InternalService;
+import org.apache.doris.proto.InternalService.PConstantExprResult;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.rpc.BackendServiceProxy;
+import org.apache.doris.system.Backend;
+import org.apache.doris.thrift.TExpr;
+import org.apache.doris.thrift.TFoldConstantParams;
+import org.apache.doris.thrift.TNetworkAddress;
+import org.apache.doris.thrift.TPrimitiveType;
+import org.apache.doris.thrift.TQueryGlobals;
+
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.text.DateFormat;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Future;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
+
+/**
+ * Constant evaluation of an expression.
+ */
+public class FoldConstantRule extends AbstractExpressionRewriteRule {
+ public static final FoldConstantRule INSTANCE = new FoldConstantRule();
+ private static final Logger LOG = LogManager.getLogger(FoldConstantRule.class);
+ private static final DateFormat DATE_FORMAT = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
+ private final IdGenerator<ExprId> idGenerator = ExprId.createGenerator();
+
+ private static boolean isAllLiteral(Expression... children) {
+ return Arrays.stream(children).allMatch(c -> c instanceof Literal);
+ }
+
+ private static boolean isAllLiteral(List<Expression> children) {
+ return children.stream().allMatch(c -> c instanceof Literal);
+ }
+
+ private static boolean hasNull(List<Expression> children) {
+ return children.stream().anyMatch(c -> c instanceof NullLiteral);
+ }
+
+ private static boolean hasNull(Expression... children) {
+ return Arrays.stream(children).anyMatch(c -> c instanceof NullLiteral);
+ }
+
+ @Override
+ public Expression rewrite(Expression expr, ExpressionRewriteContext ctx) {
+ Expression expression = expr.accept(this, ctx);
+ if (ctx.connectContext != null && ctx.connectContext.getSessionVariable().isEnableFoldConstantByBe()) {
+ return foldByBe(expression, ctx);
+ }
+ return expression;
+ }
+
+ @Override
+ public Expression visitComparisonPredicate(ComparisonPredicate cp, ExpressionRewriteContext context) {
+ Expression left = rewrite(cp.left(), context);
+ Expression right = rewrite(cp.right(), context);
+
+ if (!(cp instanceof NullSafeEqual) && hasNull(left, right)) {
+ return Literal.of(null);
+ }
+
+ if (isAllLiteral(left, right)) {
+ Literal l = (Literal) left;
+ Literal r = (Literal) right;
+ if (cp instanceof EqualTo) {
+ return BooleanLiteral.of(l.compareTo(r) == 0);
+ }
+ if (cp instanceof GreaterThan) {
+ return BooleanLiteral.of(l.compareTo(r) > 0);
+ }
+ if (cp instanceof GreaterThanEqual) {
+ return BooleanLiteral.of(l.compareTo(r) >= 0);
+ }
+ if (cp instanceof LessThan) {
+ return BooleanLiteral.of(l.compareTo(r) < 0);
+ }
+ if (cp instanceof LessThanEqual) {
+ return BooleanLiteral.of(l.compareTo(r) <= 0);
+ }
+ if (cp instanceof NullSafeEqual) {
Review Comment:
should override visitXxx too
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rewrite/rules/FoldConstantRule.java:
##########
@@ -0,0 +1,440 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.expression.rewrite.rules;
+
+import org.apache.doris.analysis.BetweenPredicate;
+import org.apache.doris.analysis.CastExpr;
+import org.apache.doris.analysis.Expr;
+import org.apache.doris.analysis.ExprId;
+import org.apache.doris.analysis.LiteralExpr;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.PrimitiveType;
+import org.apache.doris.catalog.Type;
+import org.apache.doris.common.IdGenerator;
+import org.apache.doris.common.LoadException;
+import org.apache.doris.common.util.TimeUtils;
+import org.apache.doris.common.util.VectorizedUtil;
+import org.apache.doris.nereids.glue.translator.ExpressionTranslator;
+import org.apache.doris.nereids.rules.expression.rewrite.AbstractExpressionRewriteRule;
+import org.apache.doris.nereids.rules.expression.rewrite.ExpressionRewriteContext;
+import org.apache.doris.nereids.trees.expressions.And;
+import org.apache.doris.nereids.trees.expressions.BinaryArithmetic;
+import org.apache.doris.nereids.trees.expressions.CaseWhen;
+import org.apache.doris.nereids.trees.expressions.Cast;
+import org.apache.doris.nereids.trees.expressions.ComparisonPredicate;
+import org.apache.doris.nereids.trees.expressions.CompoundPredicate;
+import org.apache.doris.nereids.trees.expressions.EqualTo;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.ExpressionEvaluator;
+import org.apache.doris.nereids.trees.expressions.GreaterThan;
+import org.apache.doris.nereids.trees.expressions.GreaterThanEqual;
+import org.apache.doris.nereids.trees.expressions.InPredicate;
+import org.apache.doris.nereids.trees.expressions.IsNull;
+import org.apache.doris.nereids.trees.expressions.LessThan;
+import org.apache.doris.nereids.trees.expressions.LessThanEqual;
+import org.apache.doris.nereids.trees.expressions.Like;
+import org.apache.doris.nereids.trees.expressions.Not;
+import org.apache.doris.nereids.trees.expressions.NullSafeEqual;
+import org.apache.doris.nereids.trees.expressions.Or;
+import org.apache.doris.nereids.trees.expressions.TimestampArithmetic;
+import org.apache.doris.nereids.trees.expressions.WhenClause;
+import org.apache.doris.nereids.trees.expressions.functions.BoundFunction;
+import org.apache.doris.nereids.trees.expressions.literal.BooleanLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.Literal;
+import org.apache.doris.nereids.trees.expressions.literal.NullLiteral;
+import org.apache.doris.nereids.types.DataType;
+import org.apache.doris.proto.InternalService;
+import org.apache.doris.proto.InternalService.PConstantExprResult;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.rpc.BackendServiceProxy;
+import org.apache.doris.system.Backend;
+import org.apache.doris.thrift.TExpr;
+import org.apache.doris.thrift.TFoldConstantParams;
+import org.apache.doris.thrift.TNetworkAddress;
+import org.apache.doris.thrift.TPrimitiveType;
+import org.apache.doris.thrift.TQueryGlobals;
+
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.text.DateFormat;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Future;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
+
+/**
+ * Constant evaluation of an expression.
+ */
+public class FoldConstantRule extends AbstractExpressionRewriteRule {
+ public static final FoldConstantRule INSTANCE = new FoldConstantRule();
+ private static final Logger LOG = LogManager.getLogger(FoldConstantRule.class);
+ private static final DateFormat DATE_FORMAT = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
+ private final IdGenerator<ExprId> idGenerator = ExprId.createGenerator();
+
+ private static boolean isAllLiteral(Expression... children) {
+ return Arrays.stream(children).allMatch(c -> c instanceof Literal);
+ }
+
+ private static boolean isAllLiteral(List<Expression> children) {
+ return children.stream().allMatch(c -> c instanceof Literal);
+ }
+
+ private static boolean hasNull(List<Expression> children) {
+ return children.stream().anyMatch(c -> c instanceof NullLiteral);
+ }
+
+ private static boolean hasNull(Expression... children) {
+ return Arrays.stream(children).anyMatch(c -> c instanceof NullLiteral);
+ }
+
+ @Override
+ public Expression rewrite(Expression expr, ExpressionRewriteContext ctx) {
+ Expression expression = expr.accept(this, ctx);
+ if (ctx.connectContext != null && ctx.connectContext.getSessionVariable().isEnableFoldConstantByBe()) {
+ return foldByBe(expression, ctx);
+ }
+ return expression;
+ }
+
+ @Override
+ public Expression visitComparisonPredicate(ComparisonPredicate cp, ExpressionRewriteContext context) {
+ Expression left = rewrite(cp.left(), context);
+ Expression right = rewrite(cp.right(), context);
+
+ if (!(cp instanceof NullSafeEqual) && hasNull(left, right)) {
+ return Literal.of(null);
+ }
+
+ if (isAllLiteral(left, right)) {
+ Literal l = (Literal) left;
+ Literal r = (Literal) right;
+ if (cp instanceof EqualTo) {
+ return BooleanLiteral.of(l.compareTo(r) == 0);
+ }
+ if (cp instanceof GreaterThan) {
+ return BooleanLiteral.of(l.compareTo(r) > 0);
+ }
+ if (cp instanceof GreaterThanEqual) {
+ return BooleanLiteral.of(l.compareTo(r) >= 0);
+ }
+ if (cp instanceof LessThan) {
+ return BooleanLiteral.of(l.compareTo(r) < 0);
+ }
+ if (cp instanceof LessThanEqual) {
+ return BooleanLiteral.of(l.compareTo(r) <= 0);
+ }
+ if (cp instanceof NullSafeEqual) {
+ if (l.isNull() && r.isNull()) {
+ return BooleanLiteral.TRUE;
+ } else if (!l.isNull() && !r.isNull()) {
+ return BooleanLiteral.of(l.compareTo(r) == 0);
+ } else {
+ return BooleanLiteral.FALSE;
+ }
+ }
+ }
+ return cp.withChildren(left, right);
+ }
+
+ @Override
+ public Expression visitNot(Not not, ExpressionRewriteContext context) {
+ Expression child = rewrite(not.child(), context);
+ if (child instanceof NullLiteral) {
+ return Literal.of(null);
+ }
+ if (child.isLiteral()) {
+ if (child instanceof BooleanLiteral) {
+ BooleanLiteral c = (BooleanLiteral) child;
+ return BooleanLiteral.of(!c.getValue());
+ }
+ }
+ return not.withChildren(child);
+ }
+
+ @Override
+ public Expression visitCompoundPredicate(CompoundPredicate compoundPredicate, ExpressionRewriteContext context) {
+ Expression left = rewrite(compoundPredicate.left(), context);
+ Expression right = rewrite(compoundPredicate.right(), context);
+ if (left instanceof NullLiteral && right instanceof NullLiteral) {
+ return Literal.of(null);
+ }
+ if (left instanceof Literal || right instanceof Literal) {
+ if (compoundPredicate instanceof And) {
+ if (left.equals(BooleanLiteral.FALSE) || right.equals(BooleanLiteral.FALSE)) {
+ return BooleanLiteral.FALSE;
+ }
+ if (left.equals(BooleanLiteral.TRUE)) {
+ return right;
+ }
+ if (right.equals(BooleanLiteral.TRUE)) {
+ return left;
+ }
+ }
+ if (compoundPredicate instanceof Or) {
+ if (left.equals(BooleanLiteral.TRUE) || right.equals(BooleanLiteral.TRUE)) {
+ return BooleanLiteral.TRUE;
+ }
+ if (left.equals(BooleanLiteral.FALSE)) {
+ return right;
+ }
+ if (right.equals(BooleanLiteral.FALSE)) {
+ return left;
+ }
+ }
+ }
+ return compoundPredicate.withChildren(left, right);
+ }
+
+ @Override
+ public Expression visitLike(Like like, ExpressionRewriteContext context) {
+ Expression left = rewrite(like.left(), context);
+ if (left instanceof NullLiteral) {
+ return Literal.of(null);
+ }
+ return like.withChildren(left, like.right());
+ }
+
+ @Override
+ public Expression visitCast(Cast cast, ExpressionRewriteContext context) {
+ if (hasNull(cast.children())) {
+ return Literal.of(null);
+ }
+ Expression child = rewrite(cast.child(), context);
+ if (child.isLiteral()) {
+ return child.castTo(cast.getDataType());
+ }
+ return cast;
+ }
+
+ @Override
+ public Expression visitBoundFunction(BoundFunction boundFunction, ExpressionRewriteContext context) {
+ List<Expression> newArgs = boundFunction.getArguments().stream().map(arg -> rewrite(arg, context))
+ .collect(Collectors.toList());
+ if (isAllLiteral(newArgs)) {
+ return ExpressionEvaluator.INSTANCE.eval(boundFunction.withChildren(newArgs));
+ }
+ return boundFunction.withChildren(newArgs);
+ }
+
+ @Override
+ public Expression visitBinaryArithmetic(BinaryArithmetic binaryArithmetic, ExpressionRewriteContext context) {
+ Expression left = rewrite(binaryArithmetic.left(), context);
+ Expression right = rewrite(binaryArithmetic.right(), context);
+ if (left instanceof NullLiteral || right instanceof NullLiteral) {
+ return Literal.of(null);
+ }
+ if (isAllLiteral(left, right)) {
+ return ExpressionEvaluator.INSTANCE.eval(binaryArithmetic.withChildren(left, right));
+ }
+ return binaryArithmetic.withChildren(left, right);
+ }
+
+ @Override
+ public Expression visitCaseWhen(CaseWhen caseWhen, ExpressionRewriteContext context) {
+ Expression newDefault = null;
+ boolean foundNewDefault = false;
+
+ List<WhenClause> whenClauses = new ArrayList<>();
+ for (WhenClause whenClause : caseWhen.getWhenClauses()) {
+ Expression whenOperand = rewrite(whenClause.getOperand(), context);
+
+ if (!(whenOperand instanceof Literal)) {
+ whenClauses.add(new WhenClause(whenOperand, rewrite(whenClause.getResult(), context)));
+ } else if (BooleanLiteral.TRUE.equals(whenOperand)) {
+ foundNewDefault = true;
+ newDefault = rewrite(whenClause.getResult(), context);
+ break;
+ }
+ }
+
+ Expression defaultResult;
+ if (foundNewDefault) {
+ defaultResult = newDefault;
+ } else {
+ defaultResult = rewrite(caseWhen.getDefaultValue().orElse(Literal.of(null)), context);
+ }
+
+ if (whenClauses.isEmpty()) {
+ return defaultResult;
+ }
+ return new CaseWhen(whenClauses, defaultResult);
+ }
+
+ @Override
+ public Expression visitInPredicate(InPredicate inPredicate, ExpressionRewriteContext context) {
+ List<Expression> newChildren = inPredicate.children().stream().map(c -> rewrite(c, context))
+ .collect(Collectors.toList());
+ if (newChildren.get(0) instanceof NullLiteral) {
+ return Literal.of(null);
+ }
+ if (isAllLiteral(newChildren)) {
+ Literal c0 = (Literal) newChildren.get(0);
+ for (int i = 1; i < newChildren.size(); i++) {
+ if (c0.compareTo((Literal) newChildren.get(i)) == 0) {
+ return Literal.of(true);
+ }
+ }
+ return Literal.of(false);
+ }
+ return inPredicate.withChildren(newChildren);
+ }
+
+ @Override
+ public Expression visitIsNull(IsNull isNull, ExpressionRewriteContext context) {
+ Expression child = rewrite(isNull.child(), context);
+ if (child instanceof NullLiteral) {
+ return Literal.of(true);
+ } else if (child instanceof Literal) {
+ return Literal.of(false);
+ }
+ return isNull.withChildren(child);
+ }
+
+ @Override
+ public Expression visitTimestampArithmetic(TimestampArithmetic arithmetic, ExpressionRewriteContext context) {
+ Expression left = rewrite(arithmetic.child(0), context);
+ Expression right = rewrite(arithmetic.child(1), context);
+ if (isAllLiteral(left, right)) {
+ return ExpressionEvaluator.INSTANCE.eval(arithmetic.withChildren(left, right));
+ }
+ return arithmetic.withChildren(left, right);
+ }
+
+ private Expression foldByBe(Expression root, ExpressionRewriteContext context) {
+ if (root.isConstant() && !root.isLiteral()) {
+ Expr expr = ExpressionTranslator.INSTANCE.translate(root, null);
+ assignId(expr);
+ Map<String, Expr> ori = new HashMap<>();
+ ori.put(expr.getId().toString(), expr);
+
+ Map<String, Map<String, TExpr>> paramMap = new HashMap<>();
+ Map<String, TExpr> constMap = new HashMap<>();
+
+ collectConst(expr, constMap);
+ paramMap.put("0", constMap);
Review Comment:
what is the "0" mean?
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rewrite/rules/FoldConstantRule.java:
##########
@@ -0,0 +1,440 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.expression.rewrite.rules;
+
+import org.apache.doris.analysis.BetweenPredicate;
+import org.apache.doris.analysis.CastExpr;
+import org.apache.doris.analysis.Expr;
+import org.apache.doris.analysis.ExprId;
+import org.apache.doris.analysis.LiteralExpr;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.PrimitiveType;
+import org.apache.doris.catalog.Type;
+import org.apache.doris.common.IdGenerator;
+import org.apache.doris.common.LoadException;
+import org.apache.doris.common.util.TimeUtils;
+import org.apache.doris.common.util.VectorizedUtil;
+import org.apache.doris.nereids.glue.translator.ExpressionTranslator;
+import org.apache.doris.nereids.rules.expression.rewrite.AbstractExpressionRewriteRule;
+import org.apache.doris.nereids.rules.expression.rewrite.ExpressionRewriteContext;
+import org.apache.doris.nereids.trees.expressions.And;
+import org.apache.doris.nereids.trees.expressions.BinaryArithmetic;
+import org.apache.doris.nereids.trees.expressions.CaseWhen;
+import org.apache.doris.nereids.trees.expressions.Cast;
+import org.apache.doris.nereids.trees.expressions.ComparisonPredicate;
+import org.apache.doris.nereids.trees.expressions.CompoundPredicate;
+import org.apache.doris.nereids.trees.expressions.EqualTo;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.ExpressionEvaluator;
+import org.apache.doris.nereids.trees.expressions.GreaterThan;
+import org.apache.doris.nereids.trees.expressions.GreaterThanEqual;
+import org.apache.doris.nereids.trees.expressions.InPredicate;
+import org.apache.doris.nereids.trees.expressions.IsNull;
+import org.apache.doris.nereids.trees.expressions.LessThan;
+import org.apache.doris.nereids.trees.expressions.LessThanEqual;
+import org.apache.doris.nereids.trees.expressions.Like;
+import org.apache.doris.nereids.trees.expressions.Not;
+import org.apache.doris.nereids.trees.expressions.NullSafeEqual;
+import org.apache.doris.nereids.trees.expressions.Or;
+import org.apache.doris.nereids.trees.expressions.TimestampArithmetic;
+import org.apache.doris.nereids.trees.expressions.WhenClause;
+import org.apache.doris.nereids.trees.expressions.functions.BoundFunction;
+import org.apache.doris.nereids.trees.expressions.literal.BooleanLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.Literal;
+import org.apache.doris.nereids.trees.expressions.literal.NullLiteral;
+import org.apache.doris.nereids.types.DataType;
+import org.apache.doris.proto.InternalService;
+import org.apache.doris.proto.InternalService.PConstantExprResult;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.rpc.BackendServiceProxy;
+import org.apache.doris.system.Backend;
+import org.apache.doris.thrift.TExpr;
+import org.apache.doris.thrift.TFoldConstantParams;
+import org.apache.doris.thrift.TNetworkAddress;
+import org.apache.doris.thrift.TPrimitiveType;
+import org.apache.doris.thrift.TQueryGlobals;
+
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.text.DateFormat;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Future;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
+
+/**
+ * Constant evaluation of an expression.
+ */
+public class FoldConstantRule extends AbstractExpressionRewriteRule {
+ public static final FoldConstantRule INSTANCE = new FoldConstantRule();
+ private static final Logger LOG = LogManager.getLogger(FoldConstantRule.class);
+ private static final DateFormat DATE_FORMAT = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
+ private final IdGenerator<ExprId> idGenerator = ExprId.createGenerator();
+
+ private static boolean isAllLiteral(Expression... children) {
+ return Arrays.stream(children).allMatch(c -> c instanceof Literal);
+ }
+
+ private static boolean isAllLiteral(List<Expression> children) {
+ return children.stream().allMatch(c -> c instanceof Literal);
+ }
+
Review Comment:
this static methods should move to ExpressionUtils
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/literal/Literal.java:
##########
@@ -82,6 +107,90 @@ public <R, C> R accept(ExpressionVisitor<R, C> visitor, C context) {
return visitor.visitLiteral(this, context);
}
+ public boolean isNull() {
+ return this instanceof NullLiteral;
+ }
+
+ /**
+ * literal expr compare.
+ */
+ public int compareTo(Literal other) {
+ if (isNull() && other.isNull()) {
+ return 0;
+ }
+ DataType oType = other.getDataType();
+ DataType type = getDataType();
+ if (!type.equals(oType)) {
+ throw new RuntimeException("data type not equal!");
+ } else if (type.isBooleanType()) {
+ return Boolean.compare((boolean) getValue(), (boolean) other.getValue());
+ } else if (type.isTinyIntType()) {
+ return Byte.compare((byte) getValue(), (byte) other.getValue());
+ } else if (type.isSmallIntType()) {
+ return Short.compare((short) getValue(), (short) other.getValue());
+ } else if (type.isIntType()) {
+ return Integer.compare((int) getValue(), (int) other.getValue());
+ } else if (type.isBigIntType()) {
+ return Long.compare((long) getValue(), (long) other.getValue());
+ } else if (type.isLargeIntType()) {
+ return ((BigInteger) getValue()).compareTo((BigInteger) other.getValue());
+ } else if (type.isFloatType()) {
+ return Float.compare((float) getValue(), (float) other.getValue());
+ } else if (type.isDoubleType()) {
+ return Double.compare((double) getValue(), (double) other.getValue());
+ } else if (type.isDecimalType()) {
+ return Long.compare((Long) getValue(), (Long) other.getValue());
+ } else if (type.isDateType()) {
+ // todo process date
+ } else if (type.isDecimalType()) {
+ return ((BigDecimal) getValue()).compareTo((BigDecimal) other.getValue());
+ } else if (type instanceof StringType) {
+ return StringUtils.compare((String) getValue(), (String) other.getValue());
+ }
+ return -1;
+ }
+
+ @Override
+ protected Expression uncheckedCastTo(DataType targetType) throws AnalysisException {
+ String desc = getStringValue();
+ if (targetType.isBooleanType()) {
+ if ("0".equals(desc) || "false".equals(desc.toLowerCase(Locale.ROOT))) {
+ return Literal.of(false);
+ }
+ if ("1".equals(desc) || "true".equals(desc.toLowerCase(Locale.ROOT))) {
+ return Literal.of(true);
+ }
+ }
+ if (targetType.isTinyIntType()) {
+ return Literal.of(Double.valueOf(desc).byteValue());
+ } else if (targetType.isSmallIntType()) {
+ return Literal.of(Double.valueOf(desc).shortValue());
+ } else if (targetType.isIntType()) {
+ return Literal.of(Double.valueOf(desc).intValue());
+ } else if (targetType.isBigIntType()) {
+ return Literal.of(Double.valueOf(desc).longValue());
+ } else if (targetType.isLargeIntType()) {
+ return Literal.of(new BigInteger(desc));
+ } else if (targetType.isFloatType()) {
+ return Literal.of(Float.parseFloat(desc));
+ } else if (targetType.isDoubleType()) {
+ return Literal.of(Double.parseDouble(desc));
+ } else if (targetType.isCharType()) {
+ return new CharLiteral(desc, ((CharType) targetType).getLen());
+ } else if (targetType.isVarcharType()) {
+ return new VarcharLiteral(desc, ((VarcharType) targetType).getLen());
+ } else if (targetType.isStringType()) {
+ return Literal.of(desc);
+ } else if (targetType.isDate()) {
+ return new DateLiteral(desc);
+ } else if (targetType.isDateTime()) {
+ return new DateTimeLiteral(desc);
+ } else if (targetType.isDecimalType()) {
+ return new DecimalLiteral(BigDecimal.valueOf(Double.parseDouble(desc)));
Review Comment:
ditto
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/literal/Literal.java:
##########
@@ -82,6 +107,90 @@ public <R, C> R accept(ExpressionVisitor<R, C> visitor, C context) {
return visitor.visitLiteral(this, context);
}
+ public boolean isNull() {
+ return this instanceof NullLiteral;
+ }
+
+ /**
+ * literal expr compare.
+ */
+ public int compareTo(Literal other) {
+ if (isNull() && other.isNull()) {
+ return 0;
+ }
+ DataType oType = other.getDataType();
+ DataType type = getDataType();
+ if (!type.equals(oType)) {
+ throw new RuntimeException("data type not equal!");
+ } else if (type.isBooleanType()) {
+ return Boolean.compare((boolean) getValue(), (boolean) other.getValue());
+ } else if (type.isTinyIntType()) {
+ return Byte.compare((byte) getValue(), (byte) other.getValue());
+ } else if (type.isSmallIntType()) {
+ return Short.compare((short) getValue(), (short) other.getValue());
+ } else if (type.isIntType()) {
+ return Integer.compare((int) getValue(), (int) other.getValue());
+ } else if (type.isBigIntType()) {
+ return Long.compare((long) getValue(), (long) other.getValue());
+ } else if (type.isLargeIntType()) {
+ return ((BigInteger) getValue()).compareTo((BigInteger) other.getValue());
+ } else if (type.isFloatType()) {
+ return Float.compare((float) getValue(), (float) other.getValue());
+ } else if (type.isDoubleType()) {
+ return Double.compare((double) getValue(), (double) other.getValue());
+ } else if (type.isDecimalType()) {
+ return Long.compare((Long) getValue(), (Long) other.getValue());
+ } else if (type.isDateType()) {
+ // todo process date
+ } else if (type.isDecimalType()) {
+ return ((BigDecimal) getValue()).compareTo((BigDecimal) other.getValue());
+ } else if (type instanceof StringType) {
+ return StringUtils.compare((String) getValue(), (String) other.getValue());
+ }
Review Comment:
why not compare in every Literal child class?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #12151: [Nereids][Feature] Constant fold of an expression
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #12151:
URL: https://github.com/apache/doris/pull/12151#issuecomment-1257708359
PR approved by at least one committer and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] qzsee commented on a diff in pull request #12151: [Nereids][Feature] Constant fold of an expression
Posted by GitBox <gi...@apache.org>.
qzsee commented on code in PR #12151:
URL: https://github.com/apache/doris/pull/12151#discussion_r963204841
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/literal/Literal.java:
##########
@@ -82,6 +107,90 @@ public <R, C> R accept(ExpressionVisitor<R, C> visitor, C context) {
return visitor.visitLiteral(this, context);
}
+ public boolean isNull() {
+ return this instanceof NullLiteral;
+ }
+
+ /**
+ * literal expr compare.
+ */
+ public int compareTo(Literal other) {
+ if (isNull() && other.isNull()) {
+ return 0;
+ }
+ DataType oType = other.getDataType();
+ DataType type = getDataType();
+ if (!type.equals(oType)) {
+ throw new RuntimeException("data type not equal!");
+ } else if (type.isBooleanType()) {
+ return Boolean.compare((boolean) getValue(), (boolean) other.getValue());
+ } else if (type.isTinyIntType()) {
+ return Byte.compare((byte) getValue(), (byte) other.getValue());
+ } else if (type.isSmallIntType()) {
+ return Short.compare((short) getValue(), (short) other.getValue());
+ } else if (type.isIntType()) {
+ return Integer.compare((int) getValue(), (int) other.getValue());
+ } else if (type.isBigIntType()) {
+ return Long.compare((long) getValue(), (long) other.getValue());
+ } else if (type.isLargeIntType()) {
+ return ((BigInteger) getValue()).compareTo((BigInteger) other.getValue());
+ } else if (type.isFloatType()) {
+ return Float.compare((float) getValue(), (float) other.getValue());
+ } else if (type.isDoubleType()) {
+ return Double.compare((double) getValue(), (double) other.getValue());
+ } else if (type.isDecimalType()) {
+ return Long.compare((Long) getValue(), (Long) other.getValue());
+ } else if (type.isDateType()) {
+ // todo process date
+ } else if (type.isDecimalType()) {
+ return ((BigDecimal) getValue()).compareTo((BigDecimal) other.getValue());
+ } else if (type instanceof StringType) {
+ return StringUtils.compare((String) getValue(), (String) other.getValue());
+ }
Review Comment:
Literal generic comparison functions, so if you put them in each literal there's redundant code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] 924060929 commented on a diff in pull request #12151: [Nereids][Feature] Constant fold of an expression
Posted by GitBox <gi...@apache.org>.
924060929 commented on code in PR #12151:
URL: https://github.com/apache/doris/pull/12151#discussion_r959230265
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rewrite/rules/FoldConstantRule.java:
##########
@@ -0,0 +1,440 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.expression.rewrite.rules;
+
+import org.apache.doris.analysis.BetweenPredicate;
+import org.apache.doris.analysis.CastExpr;
+import org.apache.doris.analysis.Expr;
+import org.apache.doris.analysis.ExprId;
+import org.apache.doris.analysis.LiteralExpr;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.PrimitiveType;
+import org.apache.doris.catalog.Type;
+import org.apache.doris.common.IdGenerator;
+import org.apache.doris.common.LoadException;
+import org.apache.doris.common.util.TimeUtils;
+import org.apache.doris.common.util.VectorizedUtil;
+import org.apache.doris.nereids.glue.translator.ExpressionTranslator;
+import org.apache.doris.nereids.rules.expression.rewrite.AbstractExpressionRewriteRule;
+import org.apache.doris.nereids.rules.expression.rewrite.ExpressionRewriteContext;
+import org.apache.doris.nereids.trees.expressions.And;
+import org.apache.doris.nereids.trees.expressions.BinaryArithmetic;
+import org.apache.doris.nereids.trees.expressions.CaseWhen;
+import org.apache.doris.nereids.trees.expressions.Cast;
+import org.apache.doris.nereids.trees.expressions.ComparisonPredicate;
+import org.apache.doris.nereids.trees.expressions.CompoundPredicate;
+import org.apache.doris.nereids.trees.expressions.EqualTo;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.ExpressionEvaluator;
+import org.apache.doris.nereids.trees.expressions.GreaterThan;
+import org.apache.doris.nereids.trees.expressions.GreaterThanEqual;
+import org.apache.doris.nereids.trees.expressions.InPredicate;
+import org.apache.doris.nereids.trees.expressions.IsNull;
+import org.apache.doris.nereids.trees.expressions.LessThan;
+import org.apache.doris.nereids.trees.expressions.LessThanEqual;
+import org.apache.doris.nereids.trees.expressions.Like;
+import org.apache.doris.nereids.trees.expressions.Not;
+import org.apache.doris.nereids.trees.expressions.NullSafeEqual;
+import org.apache.doris.nereids.trees.expressions.Or;
+import org.apache.doris.nereids.trees.expressions.TimestampArithmetic;
+import org.apache.doris.nereids.trees.expressions.WhenClause;
+import org.apache.doris.nereids.trees.expressions.functions.BoundFunction;
+import org.apache.doris.nereids.trees.expressions.literal.BooleanLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.Literal;
+import org.apache.doris.nereids.trees.expressions.literal.NullLiteral;
+import org.apache.doris.nereids.types.DataType;
+import org.apache.doris.proto.InternalService;
+import org.apache.doris.proto.InternalService.PConstantExprResult;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.rpc.BackendServiceProxy;
+import org.apache.doris.system.Backend;
+import org.apache.doris.thrift.TExpr;
+import org.apache.doris.thrift.TFoldConstantParams;
+import org.apache.doris.thrift.TNetworkAddress;
+import org.apache.doris.thrift.TPrimitiveType;
+import org.apache.doris.thrift.TQueryGlobals;
+
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.text.DateFormat;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Future;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
+
+/**
+ * Constant evaluation of an expression.
+ */
+public class FoldConstantRule extends AbstractExpressionRewriteRule {
+ public static final FoldConstantRule INSTANCE = new FoldConstantRule();
+ private static final Logger LOG = LogManager.getLogger(FoldConstantRule.class);
+ private static final DateFormat DATE_FORMAT = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
+ private final IdGenerator<ExprId> idGenerator = ExprId.createGenerator();
+
+ private static boolean isAllLiteral(Expression... children) {
+ return Arrays.stream(children).allMatch(c -> c instanceof Literal);
+ }
+
+ private static boolean isAllLiteral(List<Expression> children) {
+ return children.stream().allMatch(c -> c instanceof Literal);
+ }
+
+ private static boolean hasNull(List<Expression> children) {
+ return children.stream().anyMatch(c -> c instanceof NullLiteral);
+ }
+
+ private static boolean hasNull(Expression... children) {
+ return Arrays.stream(children).anyMatch(c -> c instanceof NullLiteral);
+ }
+
+ @Override
+ public Expression rewrite(Expression expr, ExpressionRewriteContext ctx) {
+ Expression expression = expr.accept(this, ctx);
+ if (ctx.connectContext != null && ctx.connectContext.getSessionVariable().isEnableFoldConstantByBe()) {
+ return foldByBe(expression, ctx);
+ }
+ return expression;
+ }
+
+ @Override
+ public Expression visitComparisonPredicate(ComparisonPredicate cp, ExpressionRewriteContext context) {
+ Expression left = rewrite(cp.left(), context);
+ Expression right = rewrite(cp.right(), context);
+
+ if (!(cp instanceof NullSafeEqual) && hasNull(left, right)) {
+ return Literal.of(null);
+ }
+
+ if (isAllLiteral(left, right)) {
+ Literal l = (Literal) left;
+ Literal r = (Literal) right;
+ if (cp instanceof EqualTo) {
+ return BooleanLiteral.of(l.compareTo(r) == 0);
+ }
+ if (cp instanceof GreaterThan) {
+ return BooleanLiteral.of(l.compareTo(r) > 0);
+ }
+ if (cp instanceof GreaterThanEqual) {
+ return BooleanLiteral.of(l.compareTo(r) >= 0);
+ }
+ if (cp instanceof LessThan) {
+ return BooleanLiteral.of(l.compareTo(r) < 0);
+ }
+ if (cp instanceof LessThanEqual) {
+ return BooleanLiteral.of(l.compareTo(r) <= 0);
+ }
+ if (cp instanceof NullSafeEqual) {
+ if (l.isNull() && r.isNull()) {
+ return BooleanLiteral.TRUE;
+ } else if (!l.isNull() && !r.isNull()) {
+ return BooleanLiteral.of(l.compareTo(r) == 0);
+ } else {
+ return BooleanLiteral.FALSE;
+ }
+ }
+ }
+ return cp.withChildren(left, right);
+ }
+
+ @Override
+ public Expression visitNot(Not not, ExpressionRewriteContext context) {
+ Expression child = rewrite(not.child(), context);
+ if (child instanceof NullLiteral) {
+ return Literal.of(null);
+ }
Review Comment:
this part look like a common logic, you can move to the common visit method, like this:
```java
public Expression visit(Expression expr, ExpressionRewriteContext context) {
List<Expression> children = expr.children()
.stream()
.map(child -> child.visit(this, context)
.collect(Collectors.toList());
boolean hasNullLiteralChild = ExpresstionUtils.hasNullLiteral(children);
// isNullHandling: Env.getCurrentEnv().isNullResultWithOneNullParamFunction()
if (!expr.isNullHandling() && hasNullLiteralChild) {
return Literal.of(null);
}
if (!ExpressionUtils.isAllLiteral(children) {
return expr.withChildren(children);
}
// this position every child is Literal, and forward to visitXxx method
return expr.withChildren(children).accept(this, context);
}
```
so every visitXxx method can simple fold constant because every child must be Literal
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] qzsee commented on a diff in pull request #12151: [Nereids][Feature] Constant fold of an expression
Posted by GitBox <gi...@apache.org>.
qzsee commented on code in PR #12151:
URL: https://github.com/apache/doris/pull/12151#discussion_r978227682
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rewrite/rules/FoldConstantRuleOnFE.java:
##########
@@ -0,0 +1,310 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.expression.rewrite.rules;
+
+import org.apache.doris.nereids.rules.expression.rewrite.AbstractExpressionRewriteRule;
+import org.apache.doris.nereids.rules.expression.rewrite.ExpressionRewriteContext;
+import org.apache.doris.nereids.trees.expressions.And;
+import org.apache.doris.nereids.trees.expressions.BinaryArithmetic;
+import org.apache.doris.nereids.trees.expressions.CaseWhen;
+import org.apache.doris.nereids.trees.expressions.Cast;
+import org.apache.doris.nereids.trees.expressions.EqualTo;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.ExpressionEvaluator;
+import org.apache.doris.nereids.trees.expressions.GreaterThan;
+import org.apache.doris.nereids.trees.expressions.GreaterThanEqual;
+import org.apache.doris.nereids.trees.expressions.InPredicate;
+import org.apache.doris.nereids.trees.expressions.IsNull;
+import org.apache.doris.nereids.trees.expressions.LessThan;
+import org.apache.doris.nereids.trees.expressions.LessThanEqual;
+import org.apache.doris.nereids.trees.expressions.Like;
+import org.apache.doris.nereids.trees.expressions.Not;
+import org.apache.doris.nereids.trees.expressions.NullSafeEqual;
+import org.apache.doris.nereids.trees.expressions.Or;
+import org.apache.doris.nereids.trees.expressions.Slot;
+import org.apache.doris.nereids.trees.expressions.TimestampArithmetic;
+import org.apache.doris.nereids.trees.expressions.WhenClause;
+import org.apache.doris.nereids.trees.expressions.functions.BoundFunction;
+import org.apache.doris.nereids.trees.expressions.functions.PropagateNullable;
+import org.apache.doris.nereids.trees.expressions.literal.BooleanLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.Literal;
+import org.apache.doris.nereids.util.ExpressionUtils;
+
+import com.google.common.collect.Lists;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * evaluate an expression on fe.
+ */
+public class FoldConstantRuleOnFE extends AbstractExpressionRewriteRule {
+ public static final FoldConstantRuleOnFE INSTANCE = new FoldConstantRuleOnFE();
+
+ @Override
+ public Expression rewrite(Expression expr, ExpressionRewriteContext ctx) {
+ return process(expr, ctx);
+ }
+
+ @Override
+ public Expression visit(Expression expr, ExpressionRewriteContext context) {
+ return expr;
+ }
+
+ /**
+ * process constant expression.
+ */
+ public Expression process(Expression expr, ExpressionRewriteContext ctx) {
+ if (expr instanceof PropagateNullable) {
Review Comment:
When you eval an expression, you first eval its children, and then you can eval itself
for example
a > 1 + 2
first eval 1 + 2 -> 3
then eval a > 3
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] qzsee commented on a diff in pull request #12151: [Nereids][Feature] Constant fold of an expression
Posted by GitBox <gi...@apache.org>.
qzsee commented on code in PR #12151:
URL: https://github.com/apache/doris/pull/12151#discussion_r978227226
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rewrite/rules/FoldConstantRuleOnFE.java:
##########
@@ -0,0 +1,310 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.expression.rewrite.rules;
+
+import org.apache.doris.nereids.rules.expression.rewrite.AbstractExpressionRewriteRule;
+import org.apache.doris.nereids.rules.expression.rewrite.ExpressionRewriteContext;
+import org.apache.doris.nereids.trees.expressions.And;
+import org.apache.doris.nereids.trees.expressions.BinaryArithmetic;
+import org.apache.doris.nereids.trees.expressions.CaseWhen;
+import org.apache.doris.nereids.trees.expressions.Cast;
+import org.apache.doris.nereids.trees.expressions.EqualTo;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.ExpressionEvaluator;
+import org.apache.doris.nereids.trees.expressions.GreaterThan;
+import org.apache.doris.nereids.trees.expressions.GreaterThanEqual;
+import org.apache.doris.nereids.trees.expressions.InPredicate;
+import org.apache.doris.nereids.trees.expressions.IsNull;
+import org.apache.doris.nereids.trees.expressions.LessThan;
+import org.apache.doris.nereids.trees.expressions.LessThanEqual;
+import org.apache.doris.nereids.trees.expressions.Like;
+import org.apache.doris.nereids.trees.expressions.Not;
+import org.apache.doris.nereids.trees.expressions.NullSafeEqual;
+import org.apache.doris.nereids.trees.expressions.Or;
+import org.apache.doris.nereids.trees.expressions.Slot;
+import org.apache.doris.nereids.trees.expressions.TimestampArithmetic;
+import org.apache.doris.nereids.trees.expressions.WhenClause;
+import org.apache.doris.nereids.trees.expressions.functions.BoundFunction;
+import org.apache.doris.nereids.trees.expressions.functions.PropagateNullable;
+import org.apache.doris.nereids.trees.expressions.literal.BooleanLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.Literal;
+import org.apache.doris.nereids.util.ExpressionUtils;
+
+import com.google.common.collect.Lists;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * evaluate an expression on fe.
+ */
+public class FoldConstantRuleOnFE extends AbstractExpressionRewriteRule {
+ public static final FoldConstantRuleOnFE INSTANCE = new FoldConstantRuleOnFE();
+
+ @Override
+ public Expression rewrite(Expression expr, ExpressionRewriteContext ctx) {
+ return process(expr, ctx);
+ }
+
+ @Override
+ public Expression visit(Expression expr, ExpressionRewriteContext context) {
+ return expr;
+ }
+
+ /**
+ * process constant expression.
+ */
+ public Expression process(Expression expr, ExpressionRewriteContext ctx) {
+ if (expr instanceof PropagateNullable) {
+ List<Expression> children = expr.children()
+ .stream()
+ .map(child -> process(child, ctx))
+ .collect(Collectors.toList());
+
+ if (ExpressionUtils.hasNullLiteral(children)) {
+ return Literal.of(null);
+ }
+
+ if (!ExpressionUtils.isAllLiteral(children)) {
+ return expr.withChildren(children);
+ }
+ return expr.withChildren(children).accept(this, ctx);
+ } else {
+ return expr.accept(this, ctx);
+ }
+ }
+
+ @Override
+ public Expression visitEqualTo(EqualTo equalTo, ExpressionRewriteContext context) {
+ return BooleanLiteral.of(((Literal) equalTo.left()).compareTo((Literal) equalTo.right()) == 0);
Review Comment:
process method
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] morrySnow commented on a diff in pull request #12151: [Nereids][Feature] Constant fold of an expression
Posted by GitBox <gi...@apache.org>.
morrySnow commented on code in PR #12151:
URL: https://github.com/apache/doris/pull/12151#discussion_r979267670
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rewrite/rules/FoldConstantRuleOnFE.java:
##########
@@ -0,0 +1,310 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.expression.rewrite.rules;
+
+import org.apache.doris.nereids.rules.expression.rewrite.AbstractExpressionRewriteRule;
+import org.apache.doris.nereids.rules.expression.rewrite.ExpressionRewriteContext;
+import org.apache.doris.nereids.trees.expressions.And;
+import org.apache.doris.nereids.trees.expressions.BinaryArithmetic;
+import org.apache.doris.nereids.trees.expressions.CaseWhen;
+import org.apache.doris.nereids.trees.expressions.Cast;
+import org.apache.doris.nereids.trees.expressions.EqualTo;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.ExpressionEvaluator;
+import org.apache.doris.nereids.trees.expressions.GreaterThan;
+import org.apache.doris.nereids.trees.expressions.GreaterThanEqual;
+import org.apache.doris.nereids.trees.expressions.InPredicate;
+import org.apache.doris.nereids.trees.expressions.IsNull;
+import org.apache.doris.nereids.trees.expressions.LessThan;
+import org.apache.doris.nereids.trees.expressions.LessThanEqual;
+import org.apache.doris.nereids.trees.expressions.Like;
+import org.apache.doris.nereids.trees.expressions.Not;
+import org.apache.doris.nereids.trees.expressions.NullSafeEqual;
+import org.apache.doris.nereids.trees.expressions.Or;
+import org.apache.doris.nereids.trees.expressions.Slot;
+import org.apache.doris.nereids.trees.expressions.TimestampArithmetic;
+import org.apache.doris.nereids.trees.expressions.WhenClause;
+import org.apache.doris.nereids.trees.expressions.functions.BoundFunction;
+import org.apache.doris.nereids.trees.expressions.functions.PropagateNullable;
+import org.apache.doris.nereids.trees.expressions.literal.BooleanLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.Literal;
+import org.apache.doris.nereids.util.ExpressionUtils;
+
+import com.google.common.collect.Lists;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * evaluate an expression on fe.
+ */
+public class FoldConstantRuleOnFE extends AbstractExpressionRewriteRule {
+ public static final FoldConstantRuleOnFE INSTANCE = new FoldConstantRuleOnFE();
+
+ @Override
+ public Expression rewrite(Expression expr, ExpressionRewriteContext ctx) {
+ return process(expr, ctx);
+ }
+
+ @Override
+ public Expression visit(Expression expr, ExpressionRewriteContext context) {
+ return expr;
+ }
+
+ /**
+ * process constant expression.
+ */
+ public Expression process(Expression expr, ExpressionRewriteContext ctx) {
+ if (expr instanceof PropagateNullable) {
Review Comment:
process method only handled the expression implement PropagateNullable. is it enough?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] morrySnow merged pull request #12151: [Nereids][Feature] Constant fold of an expression
Posted by GitBox <gi...@apache.org>.
morrySnow merged PR #12151:
URL: https://github.com/apache/doris/pull/12151
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] 924060929 commented on a diff in pull request #12151: [Nereids][Feature] Constant fold of an expression
Posted by GitBox <gi...@apache.org>.
924060929 commented on code in PR #12151:
URL: https://github.com/apache/doris/pull/12151#discussion_r959230265
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rewrite/rules/FoldConstantRule.java:
##########
@@ -0,0 +1,440 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.expression.rewrite.rules;
+
+import org.apache.doris.analysis.BetweenPredicate;
+import org.apache.doris.analysis.CastExpr;
+import org.apache.doris.analysis.Expr;
+import org.apache.doris.analysis.ExprId;
+import org.apache.doris.analysis.LiteralExpr;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.PrimitiveType;
+import org.apache.doris.catalog.Type;
+import org.apache.doris.common.IdGenerator;
+import org.apache.doris.common.LoadException;
+import org.apache.doris.common.util.TimeUtils;
+import org.apache.doris.common.util.VectorizedUtil;
+import org.apache.doris.nereids.glue.translator.ExpressionTranslator;
+import org.apache.doris.nereids.rules.expression.rewrite.AbstractExpressionRewriteRule;
+import org.apache.doris.nereids.rules.expression.rewrite.ExpressionRewriteContext;
+import org.apache.doris.nereids.trees.expressions.And;
+import org.apache.doris.nereids.trees.expressions.BinaryArithmetic;
+import org.apache.doris.nereids.trees.expressions.CaseWhen;
+import org.apache.doris.nereids.trees.expressions.Cast;
+import org.apache.doris.nereids.trees.expressions.ComparisonPredicate;
+import org.apache.doris.nereids.trees.expressions.CompoundPredicate;
+import org.apache.doris.nereids.trees.expressions.EqualTo;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.ExpressionEvaluator;
+import org.apache.doris.nereids.trees.expressions.GreaterThan;
+import org.apache.doris.nereids.trees.expressions.GreaterThanEqual;
+import org.apache.doris.nereids.trees.expressions.InPredicate;
+import org.apache.doris.nereids.trees.expressions.IsNull;
+import org.apache.doris.nereids.trees.expressions.LessThan;
+import org.apache.doris.nereids.trees.expressions.LessThanEqual;
+import org.apache.doris.nereids.trees.expressions.Like;
+import org.apache.doris.nereids.trees.expressions.Not;
+import org.apache.doris.nereids.trees.expressions.NullSafeEqual;
+import org.apache.doris.nereids.trees.expressions.Or;
+import org.apache.doris.nereids.trees.expressions.TimestampArithmetic;
+import org.apache.doris.nereids.trees.expressions.WhenClause;
+import org.apache.doris.nereids.trees.expressions.functions.BoundFunction;
+import org.apache.doris.nereids.trees.expressions.literal.BooleanLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.Literal;
+import org.apache.doris.nereids.trees.expressions.literal.NullLiteral;
+import org.apache.doris.nereids.types.DataType;
+import org.apache.doris.proto.InternalService;
+import org.apache.doris.proto.InternalService.PConstantExprResult;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.rpc.BackendServiceProxy;
+import org.apache.doris.system.Backend;
+import org.apache.doris.thrift.TExpr;
+import org.apache.doris.thrift.TFoldConstantParams;
+import org.apache.doris.thrift.TNetworkAddress;
+import org.apache.doris.thrift.TPrimitiveType;
+import org.apache.doris.thrift.TQueryGlobals;
+
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.text.DateFormat;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Future;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
+
+/**
+ * Constant evaluation of an expression.
+ */
+public class FoldConstantRule extends AbstractExpressionRewriteRule {
+ public static final FoldConstantRule INSTANCE = new FoldConstantRule();
+ private static final Logger LOG = LogManager.getLogger(FoldConstantRule.class);
+ private static final DateFormat DATE_FORMAT = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
+ private final IdGenerator<ExprId> idGenerator = ExprId.createGenerator();
+
+ private static boolean isAllLiteral(Expression... children) {
+ return Arrays.stream(children).allMatch(c -> c instanceof Literal);
+ }
+
+ private static boolean isAllLiteral(List<Expression> children) {
+ return children.stream().allMatch(c -> c instanceof Literal);
+ }
+
+ private static boolean hasNull(List<Expression> children) {
+ return children.stream().anyMatch(c -> c instanceof NullLiteral);
+ }
+
+ private static boolean hasNull(Expression... children) {
+ return Arrays.stream(children).anyMatch(c -> c instanceof NullLiteral);
+ }
+
+ @Override
+ public Expression rewrite(Expression expr, ExpressionRewriteContext ctx) {
+ Expression expression = expr.accept(this, ctx);
+ if (ctx.connectContext != null && ctx.connectContext.getSessionVariable().isEnableFoldConstantByBe()) {
+ return foldByBe(expression, ctx);
+ }
+ return expression;
+ }
+
+ @Override
+ public Expression visitComparisonPredicate(ComparisonPredicate cp, ExpressionRewriteContext context) {
+ Expression left = rewrite(cp.left(), context);
+ Expression right = rewrite(cp.right(), context);
+
+ if (!(cp instanceof NullSafeEqual) && hasNull(left, right)) {
+ return Literal.of(null);
+ }
+
+ if (isAllLiteral(left, right)) {
+ Literal l = (Literal) left;
+ Literal r = (Literal) right;
+ if (cp instanceof EqualTo) {
+ return BooleanLiteral.of(l.compareTo(r) == 0);
+ }
+ if (cp instanceof GreaterThan) {
+ return BooleanLiteral.of(l.compareTo(r) > 0);
+ }
+ if (cp instanceof GreaterThanEqual) {
+ return BooleanLiteral.of(l.compareTo(r) >= 0);
+ }
+ if (cp instanceof LessThan) {
+ return BooleanLiteral.of(l.compareTo(r) < 0);
+ }
+ if (cp instanceof LessThanEqual) {
+ return BooleanLiteral.of(l.compareTo(r) <= 0);
+ }
+ if (cp instanceof NullSafeEqual) {
+ if (l.isNull() && r.isNull()) {
+ return BooleanLiteral.TRUE;
+ } else if (!l.isNull() && !r.isNull()) {
+ return BooleanLiteral.of(l.compareTo(r) == 0);
+ } else {
+ return BooleanLiteral.FALSE;
+ }
+ }
+ }
+ return cp.withChildren(left, right);
+ }
+
+ @Override
+ public Expression visitNot(Not not, ExpressionRewriteContext context) {
+ Expression child = rewrite(not.child(), context);
+ if (child instanceof NullLiteral) {
+ return Literal.of(null);
+ }
Review Comment:
this part look like a common logic, you can move to the common visit method, like this:
```java
public Expression visit(Expression expr, ExpressionRewriteContext context) {
List<Expression> children = expr.children()
.stream()
.map(child -> child.visit(this, context)
.collect(Collectors.toList());
boolean hasNullLiteralChild = ExpresstionUtils.hasNullLiteral(children);
// isDeterminate: Env.getCurrentEnv().isNullResultWithOneNullParamFunction()
if (expr.isDeterminate() && hasNullLiteralChild) {
return Literal.of(null);
}
if (!ExpressionUtils.isAllLiteral(children) {
return expr.withChildren(children);
}
// this position every child is Literal, and forward to visitXxx method
return expr.withChildren(children).accept(this, context);
}
```
so every visitXxx method can simple fold constant because every child must be Literal
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] 924060929 commented on a diff in pull request #12151: [Nereids][Feature] Constant fold of an expression
Posted by GitBox <gi...@apache.org>.
924060929 commented on code in PR #12151:
URL: https://github.com/apache/doris/pull/12151#discussion_r959230265
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rewrite/rules/FoldConstantRule.java:
##########
@@ -0,0 +1,440 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.expression.rewrite.rules;
+
+import org.apache.doris.analysis.BetweenPredicate;
+import org.apache.doris.analysis.CastExpr;
+import org.apache.doris.analysis.Expr;
+import org.apache.doris.analysis.ExprId;
+import org.apache.doris.analysis.LiteralExpr;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.PrimitiveType;
+import org.apache.doris.catalog.Type;
+import org.apache.doris.common.IdGenerator;
+import org.apache.doris.common.LoadException;
+import org.apache.doris.common.util.TimeUtils;
+import org.apache.doris.common.util.VectorizedUtil;
+import org.apache.doris.nereids.glue.translator.ExpressionTranslator;
+import org.apache.doris.nereids.rules.expression.rewrite.AbstractExpressionRewriteRule;
+import org.apache.doris.nereids.rules.expression.rewrite.ExpressionRewriteContext;
+import org.apache.doris.nereids.trees.expressions.And;
+import org.apache.doris.nereids.trees.expressions.BinaryArithmetic;
+import org.apache.doris.nereids.trees.expressions.CaseWhen;
+import org.apache.doris.nereids.trees.expressions.Cast;
+import org.apache.doris.nereids.trees.expressions.ComparisonPredicate;
+import org.apache.doris.nereids.trees.expressions.CompoundPredicate;
+import org.apache.doris.nereids.trees.expressions.EqualTo;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.ExpressionEvaluator;
+import org.apache.doris.nereids.trees.expressions.GreaterThan;
+import org.apache.doris.nereids.trees.expressions.GreaterThanEqual;
+import org.apache.doris.nereids.trees.expressions.InPredicate;
+import org.apache.doris.nereids.trees.expressions.IsNull;
+import org.apache.doris.nereids.trees.expressions.LessThan;
+import org.apache.doris.nereids.trees.expressions.LessThanEqual;
+import org.apache.doris.nereids.trees.expressions.Like;
+import org.apache.doris.nereids.trees.expressions.Not;
+import org.apache.doris.nereids.trees.expressions.NullSafeEqual;
+import org.apache.doris.nereids.trees.expressions.Or;
+import org.apache.doris.nereids.trees.expressions.TimestampArithmetic;
+import org.apache.doris.nereids.trees.expressions.WhenClause;
+import org.apache.doris.nereids.trees.expressions.functions.BoundFunction;
+import org.apache.doris.nereids.trees.expressions.literal.BooleanLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.Literal;
+import org.apache.doris.nereids.trees.expressions.literal.NullLiteral;
+import org.apache.doris.nereids.types.DataType;
+import org.apache.doris.proto.InternalService;
+import org.apache.doris.proto.InternalService.PConstantExprResult;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.rpc.BackendServiceProxy;
+import org.apache.doris.system.Backend;
+import org.apache.doris.thrift.TExpr;
+import org.apache.doris.thrift.TFoldConstantParams;
+import org.apache.doris.thrift.TNetworkAddress;
+import org.apache.doris.thrift.TPrimitiveType;
+import org.apache.doris.thrift.TQueryGlobals;
+
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+
+import java.text.DateFormat;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Future;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
+
+/**
+ * Constant evaluation of an expression.
+ */
+public class FoldConstantRule extends AbstractExpressionRewriteRule {
+ public static final FoldConstantRule INSTANCE = new FoldConstantRule();
+ private static final Logger LOG = LogManager.getLogger(FoldConstantRule.class);
+ private static final DateFormat DATE_FORMAT = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
+ private final IdGenerator<ExprId> idGenerator = ExprId.createGenerator();
+
+ private static boolean isAllLiteral(Expression... children) {
+ return Arrays.stream(children).allMatch(c -> c instanceof Literal);
+ }
+
+ private static boolean isAllLiteral(List<Expression> children) {
+ return children.stream().allMatch(c -> c instanceof Literal);
+ }
+
+ private static boolean hasNull(List<Expression> children) {
+ return children.stream().anyMatch(c -> c instanceof NullLiteral);
+ }
+
+ private static boolean hasNull(Expression... children) {
+ return Arrays.stream(children).anyMatch(c -> c instanceof NullLiteral);
+ }
+
+ @Override
+ public Expression rewrite(Expression expr, ExpressionRewriteContext ctx) {
+ Expression expression = expr.accept(this, ctx);
+ if (ctx.connectContext != null && ctx.connectContext.getSessionVariable().isEnableFoldConstantByBe()) {
+ return foldByBe(expression, ctx);
+ }
+ return expression;
+ }
+
+ @Override
+ public Expression visitComparisonPredicate(ComparisonPredicate cp, ExpressionRewriteContext context) {
+ Expression left = rewrite(cp.left(), context);
+ Expression right = rewrite(cp.right(), context);
+
+ if (!(cp instanceof NullSafeEqual) && hasNull(left, right)) {
+ return Literal.of(null);
+ }
+
+ if (isAllLiteral(left, right)) {
+ Literal l = (Literal) left;
+ Literal r = (Literal) right;
+ if (cp instanceof EqualTo) {
+ return BooleanLiteral.of(l.compareTo(r) == 0);
+ }
+ if (cp instanceof GreaterThan) {
+ return BooleanLiteral.of(l.compareTo(r) > 0);
+ }
+ if (cp instanceof GreaterThanEqual) {
+ return BooleanLiteral.of(l.compareTo(r) >= 0);
+ }
+ if (cp instanceof LessThan) {
+ return BooleanLiteral.of(l.compareTo(r) < 0);
+ }
+ if (cp instanceof LessThanEqual) {
+ return BooleanLiteral.of(l.compareTo(r) <= 0);
+ }
+ if (cp instanceof NullSafeEqual) {
+ if (l.isNull() && r.isNull()) {
+ return BooleanLiteral.TRUE;
+ } else if (!l.isNull() && !r.isNull()) {
+ return BooleanLiteral.of(l.compareTo(r) == 0);
+ } else {
+ return BooleanLiteral.FALSE;
+ }
+ }
+ }
+ return cp.withChildren(left, right);
+ }
+
+ @Override
+ public Expression visitNot(Not not, ExpressionRewriteContext context) {
+ Expression child = rewrite(not.child(), context);
+ if (child instanceof NullLiteral) {
+ return Literal.of(null);
+ }
Review Comment:
this part look like a common logic, you can move to the common visit method, like this:
```java
public Expression visit(Expression expr, ExpressionRewriteContext context) {
List<Expression> children = expr.children()
.stream()
.map(child -> child.visit(this, context)
.collect(Collectors.toList());
boolean hasNullLiteralChild = ExpresstionUtils.hasNullLiteral(children);
// isDeterminate: Env.getCurrentEnv().isNullResultWithOneNullParamFunction()
if (expr.isDeterminate() && hasNullLiteralChild) {
return Literal.of(null);
}
if (!ExpressionUtils.isAllLiteral(children) {
return expr;
}
// this position every child is non-null Literal, and forward to visitXxx method
return expr.withChildren(children).accept(this, context);
}
```
so every visitXxx method can simple fold constant because every child must be Literal
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] qzsee commented on a diff in pull request #12151: [Nereids][Feature] Constant fold of an expression
Posted by GitBox <gi...@apache.org>.
qzsee commented on code in PR #12151:
URL: https://github.com/apache/doris/pull/12151#discussion_r963205097
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/literal/Literal.java:
##########
@@ -82,6 +107,90 @@ public <R, C> R accept(ExpressionVisitor<R, C> visitor, C context) {
return visitor.visitLiteral(this, context);
}
+ public boolean isNull() {
+ return this instanceof NullLiteral;
+ }
+
+ /**
+ * literal expr compare.
+ */
+ public int compareTo(Literal other) {
+ if (isNull() && other.isNull()) {
+ return 0;
+ }
+ DataType oType = other.getDataType();
+ DataType type = getDataType();
+ if (!type.equals(oType)) {
+ throw new RuntimeException("data type not equal!");
+ } else if (type.isBooleanType()) {
+ return Boolean.compare((boolean) getValue(), (boolean) other.getValue());
+ } else if (type.isTinyIntType()) {
+ return Byte.compare((byte) getValue(), (byte) other.getValue());
+ } else if (type.isSmallIntType()) {
+ return Short.compare((short) getValue(), (short) other.getValue());
+ } else if (type.isIntType()) {
+ return Integer.compare((int) getValue(), (int) other.getValue());
+ } else if (type.isBigIntType()) {
+ return Long.compare((long) getValue(), (long) other.getValue());
+ } else if (type.isLargeIntType()) {
+ return ((BigInteger) getValue()).compareTo((BigInteger) other.getValue());
+ } else if (type.isFloatType()) {
+ return Float.compare((float) getValue(), (float) other.getValue());
+ } else if (type.isDoubleType()) {
+ return Double.compare((double) getValue(), (double) other.getValue());
+ } else if (type.isDecimalType()) {
+ return Long.compare((Long) getValue(), (Long) other.getValue());
+ } else if (type.isDateType()) {
+ // todo process date
+ } else if (type.isDecimalType()) {
+ return ((BigDecimal) getValue()).compareTo((BigDecimal) other.getValue());
+ } else if (type instanceof StringType) {
+ return StringUtils.compare((String) getValue(), (String) other.getValue());
+ }
+ return -1;
+ }
+
+ @Override
+ protected Expression uncheckedCastTo(DataType targetType) throws AnalysisException {
+ String desc = getStringValue();
+ if (targetType.isBooleanType()) {
+ if ("0".equals(desc) || "false".equals(desc.toLowerCase(Locale.ROOT))) {
+ return Literal.of(false);
+ }
+ if ("1".equals(desc) || "true".equals(desc.toLowerCase(Locale.ROOT))) {
+ return Literal.of(true);
+ }
+ }
+ if (targetType.isTinyIntType()) {
+ return Literal.of(Double.valueOf(desc).byteValue());
+ } else if (targetType.isSmallIntType()) {
+ return Literal.of(Double.valueOf(desc).shortValue());
+ } else if (targetType.isIntType()) {
+ return Literal.of(Double.valueOf(desc).intValue());
+ } else if (targetType.isBigIntType()) {
+ return Literal.of(Double.valueOf(desc).longValue());
+ } else if (targetType.isLargeIntType()) {
+ return Literal.of(new BigInteger(desc));
+ } else if (targetType.isFloatType()) {
+ return Literal.of(Float.parseFloat(desc));
+ } else if (targetType.isDoubleType()) {
+ return Literal.of(Double.parseDouble(desc));
+ } else if (targetType.isCharType()) {
+ return new CharLiteral(desc, ((CharType) targetType).getLen());
+ } else if (targetType.isVarcharType()) {
+ return new VarcharLiteral(desc, ((VarcharType) targetType).getLen());
+ } else if (targetType.isStringType()) {
+ return Literal.of(desc);
+ } else if (targetType.isDate()) {
+ return new DateLiteral(desc);
+ } else if (targetType.isDateTime()) {
+ return new DateTimeLiteral(desc);
+ } else if (targetType.isDecimalType()) {
+ return new DecimalLiteral(BigDecimal.valueOf(Double.parseDouble(desc)));
Review Comment:
Literal generic create functions, so if you put them in each literal there's redundant code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] qzsee commented on a diff in pull request #12151: [Nereids][Feature] Constant fold of an expression
Posted by GitBox <gi...@apache.org>.
qzsee commented on code in PR #12151:
URL: https://github.com/apache/doris/pull/12151#discussion_r963201990
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java:
##########
@@ -916,6 +917,9 @@ private Expression withPredicate(Expression valueExpression, PredicateContext ct
);
}
break;
+ case DorisParser.NULL:
+ outExpression = new IsNull(valueExpression);
Review Comment:
At the end of the method, the unified NOT processing is done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #12151: [Nereids][Feature] Constant fold of an expression
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #12151:
URL: https://github.com/apache/doris/pull/12151#issuecomment-1257708409
PR approved by anyone and no changes requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] morrySnow commented on a diff in pull request #12151: [Nereids][Feature] Constant fold of an expression
Posted by GitBox <gi...@apache.org>.
morrySnow commented on code in PR #12151:
URL: https://github.com/apache/doris/pull/12151#discussion_r979267332
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rewrite/rules/FoldConstantRuleOnFE.java:
##########
@@ -0,0 +1,310 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.rules.expression.rewrite.rules;
+
+import org.apache.doris.nereids.rules.expression.rewrite.AbstractExpressionRewriteRule;
+import org.apache.doris.nereids.rules.expression.rewrite.ExpressionRewriteContext;
+import org.apache.doris.nereids.trees.expressions.And;
+import org.apache.doris.nereids.trees.expressions.BinaryArithmetic;
+import org.apache.doris.nereids.trees.expressions.CaseWhen;
+import org.apache.doris.nereids.trees.expressions.Cast;
+import org.apache.doris.nereids.trees.expressions.EqualTo;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.ExpressionEvaluator;
+import org.apache.doris.nereids.trees.expressions.GreaterThan;
+import org.apache.doris.nereids.trees.expressions.GreaterThanEqual;
+import org.apache.doris.nereids.trees.expressions.InPredicate;
+import org.apache.doris.nereids.trees.expressions.IsNull;
+import org.apache.doris.nereids.trees.expressions.LessThan;
+import org.apache.doris.nereids.trees.expressions.LessThanEqual;
+import org.apache.doris.nereids.trees.expressions.Like;
+import org.apache.doris.nereids.trees.expressions.Not;
+import org.apache.doris.nereids.trees.expressions.NullSafeEqual;
+import org.apache.doris.nereids.trees.expressions.Or;
+import org.apache.doris.nereids.trees.expressions.Slot;
+import org.apache.doris.nereids.trees.expressions.TimestampArithmetic;
+import org.apache.doris.nereids.trees.expressions.WhenClause;
+import org.apache.doris.nereids.trees.expressions.functions.BoundFunction;
+import org.apache.doris.nereids.trees.expressions.functions.PropagateNullable;
+import org.apache.doris.nereids.trees.expressions.literal.BooleanLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.Literal;
+import org.apache.doris.nereids.util.ExpressionUtils;
+
+import com.google.common.collect.Lists;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * evaluate an expression on fe.
+ */
+public class FoldConstantRuleOnFE extends AbstractExpressionRewriteRule {
+ public static final FoldConstantRuleOnFE INSTANCE = new FoldConstantRuleOnFE();
+
+ @Override
+ public Expression rewrite(Expression expr, ExpressionRewriteContext ctx) {
+ return process(expr, ctx);
+ }
+
+ @Override
+ public Expression visit(Expression expr, ExpressionRewriteContext context) {
+ return expr;
+ }
+
+ /**
+ * process constant expression.
+ */
+ public Expression process(Expression expr, ExpressionRewriteContext ctx) {
+ if (expr instanceof PropagateNullable) {
+ List<Expression> children = expr.children()
+ .stream()
+ .map(child -> process(child, ctx))
+ .collect(Collectors.toList());
+
+ if (ExpressionUtils.hasNullLiteral(children)) {
+ return Literal.of(null);
+ }
+
+ if (!ExpressionUtils.isAllLiteral(children)) {
+ return expr.withChildren(children);
+ }
+ return expr.withChildren(children).accept(this, ctx);
+ } else {
+ return expr.accept(this, ctx);
+ }
+ }
+
+ @Override
+ public Expression visitEqualTo(EqualTo equalTo, ExpressionRewriteContext context) {
+ return BooleanLiteral.of(((Literal) equalTo.left()).compareTo((Literal) equalTo.right()) == 0);
Review Comment:
process method only handled the expression implement `PropagateNullable`. is it enough?
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rewrite/ExpressionRuleExecutor.java:
##########
@@ -33,14 +32,14 @@ public class ExpressionRuleExecutor {
private final ExpressionRewriteContext ctx;
private final List<ExpressionRewriteRule> rules;
- public ExpressionRuleExecutor(List<ExpressionRewriteRule> rules) {
+ public ExpressionRuleExecutor(List<ExpressionRewriteRule> rules, ConnectContext context) {
this.rules = rules;
- this.ctx = new ExpressionRewriteContext();
+ this.ctx = new ExpressionRewriteContext(context);
}
- public ExpressionRuleExecutor(ExpressionRewriteRule rule) {
- this.rules = Lists.newArrayList(rule);
- this.ctx = new ExpressionRewriteContext();
+ public ExpressionRuleExecutor(List<ExpressionRewriteRule> rules) {
+ this.rules = rules;
+ this.ctx = new ExpressionRewriteContext(null);
Review Comment:
```suggestion
this(rules, null);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org