You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by zh...@apache.org on 2020/02/19 12:16:54 UTC
[incubator-doris] branch master updated: Fix some function with
date type bug (#2947)
This is an automated email from the ASF dual-hosted git repository.
zhaoc pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new 147953f Fix some function with date type bug (#2947)
147953f is described below
commit 147953f09e68e98e22ffc46f8e10445dabc3440f
Author: kangkaisen <ka...@apache.org>
AuthorDate: Wed Feb 19 20:16:44 2020 +0800
Fix some function with date type bug (#2947)
The logic chain is following:
1. `date_format(if(, NULL, `dt`), '%Y%m%d')` as HASH_PARTITIONED exprs，which is not right, we should use Agg intermediate materialized slot
2. we don't use Agg intermediate materialized slot as HASH_PARTITIONED exprs, becasue
```
// the parent fragment is partitioned on the grouping exprs;
// substitute grouping exprs to reference the *output* of the agg, not the input
partitionExprs = Expr.substituteList(partitionExprs,
node.getAggInfo().getIntermediateSmap(), ctx_.getRootAnalyzer(), false);
parentPartition = DataPartition.hashPartitioned(partitionExprs);
```
the partitionExprs substitute failed。
3. partitionExprs substitute failed because partitionExprs has a casttodate child,but agg info getIntermediateSmap has a cast in datetime child.
4. The cast to date or cast to datetime child exist because `TupleIsNullPredicate` insert a `if` Expr. we don't have `if date` fn, so Doris use `if int` Expr.
5. the `date` in the `catstodate` depend on slot dt date type. the `datetime` in the `catstodatetime` depend on datetime arg type in `date_format` function.
So we could fix this issue by make if fn support date type or make date_format fn support date type
---
.../main/java/org/apache/doris/analysis/TupleIsNullPredicate.java | 5 ++---
gensrc/script/doris_builtins_functions.py | 8 ++++++++
2 files changed, 10 insertions(+), 3 deletions(-)
diff --git a/fe/src/main/java/org/apache/doris/analysis/TupleIsNullPredicate.java b/fe/src/main/java/org/apache/doris/analysis/TupleIsNullPredicate.java
index 562db08..fedf720 100644
--- a/fe/src/main/java/org/apache/doris/analysis/TupleIsNullPredicate.java
+++ b/fe/src/main/java/org/apache/doris/analysis/TupleIsNullPredicate.java
@@ -17,6 +17,7 @@
package org.apache.doris.analysis;
+import com.google.common.base.Joiner;
import org.apache.doris.common.AnalysisException;
import org.apache.doris.common.UserException;
import org.apache.doris.thrift.TExprNode;
@@ -50,8 +51,6 @@ public class TupleIsNullPredicate extends Predicate {
@Override
protected void analyzeImpl(Analyzer analyzer) throws AnalysisException {
super.analyzeImpl(analyzer);
- // analyzer = analyzer;
- // evalCost_ = tupleIds_.size() * IS_NULL_COST;
}
@Override
@@ -155,6 +154,6 @@ public class TupleIsNullPredicate extends Predicate {
@Override
public String toSqlImpl() {
- return "";
+ return "TupleIsNull(" + Joiner.on(",").join(tupleIds) + ")";
}
}
diff --git a/gensrc/script/doris_builtins_functions.py b/gensrc/script/doris_builtins_functions.py
index 919c0ff..4739399 100755
--- a/gensrc/script/doris_builtins_functions.py
+++ b/gensrc/script/doris_builtins_functions.py
@@ -102,6 +102,8 @@ visible_functions = [
# Timestamp functions
[['unix_timestamp'], 'INT', [],
'_ZN5doris18TimestampFunctions7to_unixEPN9doris_udf15FunctionContextE'],
+ [['unix_timestamp'], 'INT', ['DATE'],
+ '_ZN5doris18TimestampFunctions7to_unixEPN9doris_udf15FunctionContextERKNS1_11DateTimeValE'],
[['unix_timestamp'], 'INT', ['DATETIME'],
'_ZN5doris18TimestampFunctions7to_unixEPN9doris_udf15FunctionContextERKNS1_11DateTimeValE'],
[['unix_timestamp'], 'INT', ['VARCHAR', 'VARCHAR'],
@@ -210,6 +212,9 @@ visible_functions = [
[['str_to_date'], 'DATETIME', ['VARCHAR', 'VARCHAR'],
'_ZN5doris18TimestampFunctions11str_to_dateEPN9doris_udf'
'15FunctionContextERKNS1_9StringValES6_'],
+ [['date_format'], 'VARCHAR', ['DATE', 'VARCHAR'],
+ '_ZN5doris18TimestampFunctions11date_formatEPN9doris_udf'
+ '15FunctionContextERKNS1_11DateTimeValERKNS1_9StringValE'],
[['date_format'], 'VARCHAR', ['DATETIME', 'VARCHAR'],
'_ZN5doris18TimestampFunctions11date_formatEPN9doris_udf'
'15FunctionContextERKNS1_11DateTimeValERKNS1_9StringValE'],
@@ -438,6 +443,7 @@ visible_functions = [
[['if'], 'LARGEINT', ['BOOLEAN', 'LARGEINT', 'LARGEINT'], ''],
[['if'], 'FLOAT', ['BOOLEAN', 'FLOAT', 'FLOAT'], ''],
[['if'], 'DOUBLE', ['BOOLEAN', 'DOUBLE', 'DOUBLE'], ''],
+ [['if'], 'DATE', ['BOOLEAN', 'DATE', 'DATE'], ''],
[['if'], 'DATETIME', ['BOOLEAN', 'DATETIME', 'DATETIME'], ''],
[['if'], 'DECIMAL', ['BOOLEAN', 'DECIMAL', 'DECIMAL'], ''],
[['if'], 'DECIMALV2', ['BOOLEAN', 'DECIMALV2', 'DECIMALV2'], ''],
@@ -452,6 +458,7 @@ visible_functions = [
[['nullif'], 'LARGEINT', ['LARGEINT', 'LARGEINT'], ''],
[['nullif'], 'FLOAT', ['FLOAT', 'FLOAT'], ''],
[['nullif'], 'DOUBLE', ['DOUBLE', 'DOUBLE'], ''],
+ [['nullif'], 'DATE', ['DATE', 'DATE'], ''],
[['nullif'], 'DATETIME', ['DATETIME', 'DATETIME'], ''],
[['nullif'], 'DECIMAL', ['DECIMAL', 'DECIMAL'], ''],
[['nullif'], 'DECIMALV2', ['DECIMALV2', 'DECIMALV2'], ''],
@@ -481,6 +488,7 @@ visible_functions = [
[['coalesce'], 'LARGEINT', ['LARGEINT', '...'], ''],
[['coalesce'], 'FLOAT', ['FLOAT', '...'], ''],
[['coalesce'], 'DOUBLE', ['DOUBLE', '...'], ''],
+ [['coalesce'], 'DATE', ['DATE', '...'], ''],
[['coalesce'], 'DATETIME', ['DATETIME', '...'], ''],
[['coalesce'], 'DECIMAL', ['DECIMAL', '...'], ''],
[['coalesce'], 'DECIMALV2', ['DECIMALV2', '...'], ''],
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org