[GitHub] [druid] jon-wei commented on a change in pull request #10499: support for vectorizing expressions with non-existent inputs, more consistent type handling for non-vectorized expressions

[GitBox <gi...@apache.org>]

jon-wei commented on a change in pull request #10499:
URL: https://github.com/apache/druid/pull/10499#discussion_r510514679



##########
File path: core/src/main/java/org/apache/druid/math/expr/ExprTypeConversion.java
##########
@@ -31,22 +31,40 @@
    * Infer the output type of a list of possible 'conditional' expression outputs (where any of these could be the
    * output expression if the corresponding case matching expression evaluates to true)
    */
-  static ExprType conditional(Expr.InputBindingTypes inputTypes, List<Expr> args)
+  static ExprType conditional(Expr.InputBindingInspector inspector, List<Expr> args)
   {
     ExprType type = null;
     for (Expr arg : args) {
       if (arg.isNullLiteral()) {
         continue;
       }
       if (type == null) {
-        type = arg.getOutputType(inputTypes);
+        type = arg.getOutputType(inspector);
       } else {
-        type = doubleMathFunction(type, arg.getOutputType(inputTypes));
+        type = function(type, arg.getOutputType(inspector));
       }
     }
     return type;
   }
 
+  /**
+   * Given 2 'input' types, which might not be fully trustable, choose the most appropriate combined type for
+   * non-vectorized, per-row type detection. In this mode, null values are {@link ExprType#STRING} typed, despite
+   * potentially coming from an underlying numeric column. This method is not well suited for array handling
+   */
+  public static ExprType autoDetect(ExprEval result, ExprEval other)

Review comment:
       Can you add a part to the javadoc about when the input types would not be trustable (is it because of the string nulls from numeric columns, or are there other cases)?

##########
File path: processing/src/main/java/org/apache/druid/segment/virtual/ExpressionPlanner.java
##########
@@ -135,18 +137,29 @@ public static ExpressionPlan plan(ColumnInspector inspector, Expr expression)
         traits.add(ExpressionPlan.Trait.NON_SCALAR_INPUTS);
       }
 
-      if (!maybeMultiValued.isEmpty()) {
+      if (!noCapabilities.isEmpty()) {
         traits.add(ExpressionPlan.Trait.UNKNOWN_INPUTS);
       }
 
+      if (!maybeMultiValued.isEmpty()) {
+        traits.add(ExpressionPlan.Trait.INCOMPLETE_INPUTS);
+      }
+
       // if expression needs transformed, lets do it
       if (!needsApplied.isEmpty()) {
         traits.add(ExpressionPlan.Trait.NEEDS_APPLIED);
       }
     }
 
-    // only set output type
-    if (ExpressionPlan.none(traits, ExpressionPlan.Trait.UNKNOWN_INPUTS, ExpressionPlan.Trait.NEEDS_APPLIED)) {
+    // only set output type if we are pretty confident about input types
+    final boolean shoulComputeOutput = ExpressionPlan.none(

Review comment:
       shoulComputeOutput -> shouldComputeOutput

##########
File path: core/src/main/java/org/apache/druid/math/expr/ExprTypeConversion.java
##########
@@ -74,6 +93,15 @@ public static ExprType operator(@Nullable ExprType type, @Nullable ExprType othe
       return ExprType.STRING;
     }
 
+    // non-vectorized expressions
+    if (type == ExprType.STRING) {

Review comment:
       Can you update the javadoc for this method with the mixed type case and the reasoning behind preferring the non-string type?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org