You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2022/10/28 09:57:18 UTC

[GitHub] [doris] 924060929 opened a new pull request, #13764: [enhancement](Nereids) Support lots of scalar function and fix some bug

924060929 opened a new pull request, #13764:
URL: https://github.com/apache/doris/pull/13764

   # Proposed changes
   
   Issue Number: close #xxx
   
   ## Problem summary
   
   Describe your changes.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: 
       - [ ] Yes
       - [ ] No
       - [ ] I don't know
   2. Has unit tests been added:
       - [ ] Yes
       - [ ] No
       - [ ] No Need
   3. Has document been added or modified:
       - [ ] Yes
       - [ ] No
       - [ ] No Need
   4. Does it need to update dependencies:
       - [ ] Yes
       - [ ] No
   5. Are there any changes that cannot be rolled back:
       - [ ] Yes (If Yes, please explain WHY)
       - [ ] No
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at [dev@doris.apache.org](mailto:dev@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [doris] morrySnow commented on a diff in pull request #13764: [feature](Nereids) Support lots of scalar function and fix some bug

Posted by GitBox <gi...@apache.org>.
morrySnow commented on code in PR #13764:
URL: https://github.com/apache/doris/pull/13764#discussion_r1010205193


##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/agg/AggregateParam.java:
##########
@@ -0,0 +1,96 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.trees.expressions.functions.agg;
+
+import org.apache.doris.nereids.types.DataType;
+
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+
+/** AggregateParam. */
+public class AggregateParam {
+    public final boolean isGlobal;

Review Comment:
   isGlobal is same with isMerge in legacy planner?



##########
fe/fe-core/src/main/java/org/apache/doris/catalog/FunctionSignature.java:
##########
@@ -0,0 +1,115 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.catalog;
+
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.types.DataType;
+
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.Lists;
+
+import java.util.Arrays;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.function.BiFunction;
+
+public class FunctionSignature {
+    public final DataType returnType;
+    public final boolean hasVarArgs;
+    public final List<DataType> argumentsTypes;
+    public final int arity;
+
+    public FunctionSignature(DataType returnType, boolean hasVarArgs, List<DataType> argumentsTypes) {
+        this.returnType = Objects.requireNonNull(returnType, "returnType is not null");
+        this.argumentsTypes = ImmutableList.copyOf(
+                Objects.requireNonNull(argumentsTypes, "argumentsTypes is not null"));
+        this.hasVarArgs = hasVarArgs;
+        this.arity = argumentsTypes.size();
+    }
+
+    public Optional<DataType> getVarArgType() {
+        return hasVarArgs ? Optional.of(argumentsTypes.get(arity - 1)) : Optional.empty();
+    }
+
+    public DataType getArgType(int index) {
+        if (hasVarArgs && index >= arity) {
+            return argumentsTypes.get(arity - 1);
+        }
+        return argumentsTypes.get(index);
+    }
+
+    public FunctionSignature withReturnType(DataType returnType) {
+        return new FunctionSignature(returnType, hasVarArgs, argumentsTypes);
+    }
+
+    public FunctionSignature withArgumentTypes(boolean hasVarArgs, List<DataType> argumentsTypes) {
+        return new FunctionSignature(returnType, hasVarArgs, argumentsTypes);
+    }
+
+    /**
+     * change argument type by the signature's type and the corresponding argument's type
+     * @param arguments arguments
+     * @param transform param1: signature's type, param2: argument's type, return new type you want to change
+     * @return
+     */
+    public FunctionSignature withArgumentTypes(List<Expression> arguments,
+            BiFunction<DataType, Expression, DataType> transform) {
+        List<DataType> newTypes = Lists.newArrayList();
+        for (int i = 0; i < arguments.size(); i++) {
+            newTypes.add(transform.apply(getArgType(i), arguments.get(i)));
+        }
+        return withArgumentTypes(hasVarArgs, newTypes);
+    }
+
+    public static FunctionSignature of(DataType returnType, List<DataType> argumentsTypes) {
+        return of(returnType, false, argumentsTypes);
+    }
+
+    public static FunctionSignature of(DataType returnType, boolean hasVarArgs, List<DataType> argumentsTypes) {
+        return new FunctionSignature(returnType, hasVarArgs, argumentsTypes);
+    }
+
+    public static FunctionSignature of(DataType returnType, DataType... argumentsTypes) {
+        return of(returnType, false, argumentsTypes);
+    }
+
+    public static FunctionSignature of(DataType returnType, boolean hasVarArgs, DataType... argumentsTypes) {
+        return new FunctionSignature(returnType, hasVarArgs, Arrays.asList(argumentsTypes));
+    }
+
+    public static FuncSigBuilder ret(DataType returnType) {
+        return new FuncSigBuilder(returnType);
+    }
+
+    public static class FuncSigBuilder {

Review Comment:
   Builder with no build method is odd



##########
fe/fe-core/src/main/java/org/apache/doris/catalog/FunctionSignature.java:
##########
@@ -0,0 +1,115 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.catalog;
+
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.types.DataType;
+
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.Lists;
+
+import java.util.Arrays;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.function.BiFunction;
+
+public class FunctionSignature {
+    public final DataType returnType;
+    public final boolean hasVarArgs;
+    public final List<DataType> argumentsTypes;

Review Comment:
   ```suggestion
       public final ImmutableList<DataType> argumentsTypes;
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [doris] github-actions[bot] commented on pull request #13764: [feature](Nereids) Support lots of scalar function and fix some bug

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13764:
URL: https://github.com/apache/doris/pull/13764#issuecomment-1298348168

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [doris] hello-stephen commented on pull request #13764: [feature](Nereids) Support lots of scalar function and fix some bug

Posted by GitBox <gi...@apache.org>.
hello-stephen commented on PR #13764:
URL: https://github.com/apache/doris/pull/13764#issuecomment-1294869499

   TeamCity pipeline, clickbench performance test result:
    the sum of best hot time: 38.49 seconds
    load time: 563 seconds
    storage size: 17154699363 Bytes
    https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221028111426_clickbench_pr_35355.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [doris] hello-stephen commented on pull request #13764: [feature](Nereids) Support lots of scalar function and fix some bug

Posted by GitBox <gi...@apache.org>.
hello-stephen commented on PR #13764:
URL: https://github.com/apache/doris/pull/13764#issuecomment-1294919364

   TeamCity pipeline, clickbench performance test result:
    the sum of best hot time: 40.29 seconds
    load time: 564 seconds
    storage size: 17154644858 Bytes
    https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221028120820_clickbench_pr_35396.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [doris] hello-stephen commented on pull request #13764: [feature](Nereids) Support lots of scalar function and fix some bug

Posted by GitBox <gi...@apache.org>.
hello-stephen commented on PR #13764:
URL: https://github.com/apache/doris/pull/13764#issuecomment-1294841849

   TeamCity pipeline, clickbench performance test result:
    the sum of best hot time: 38.42 seconds
    load time: 558 seconds
    storage size: 17154644772 Bytes
    https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221028104344_clickbench_pr_35329.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [doris] 924060929 commented on a diff in pull request #13764: [feature](Nereids) Support lots of scalar function and fix some bug

Posted by GitBox <gi...@apache.org>.
924060929 commented on code in PR #13764:
URL: https://github.com/apache/doris/pull/13764#discussion_r1010246718


##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/agg/AggregateParam.java:
##########
@@ -0,0 +1,96 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.trees.expressions.functions.agg;
+
+import org.apache.doris.nereids.types.DataType;
+
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+
+/** AggregateParam. */
+public class AggregateParam {
+    public final boolean isGlobal;

Review Comment:
   yes



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [doris] 924060929 merged pull request #13764: [feature](Nereids) Support lots of scalar function and fix some bug

Posted by GitBox <gi...@apache.org>.
924060929 merged PR #13764:
URL: https://github.com/apache/doris/pull/13764


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org

[GitHub] [doris] github-actions[bot] commented on pull request #13764: [feature](Nereids) Support lots of scalar function and fix some bug

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13764:
URL: https://github.com/apache/doris/pull/13764#issuecomment-1298348122

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org