[GitHub] spark pull request #11274: [SPARK-13404] [SQL] Create variables for input ro...

[srowen <gi...@git.apache.org>]

Github user srowen commented on a diff in the pull request:

    https://github.com/apache/spark/pull/11274#discussion_r207080008
  
    --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegen.scala ---
    @@ -114,27 +117,71 @@ trait CodegenSupport extends SparkPlan {
       }
     
       /**
    -    * Consume the columns generated from it's child, call doConsume() or emit the rows.
    +    * Returns source code to evaluate all the variables, and clear the code of them, to prevent
    +    * them to be evaluated twice.
    +    */
    +  protected def evaluateVariables(variables: Seq[ExprCode]): String = {
    +    val evaluate = variables.filter(_.code != "").map(_.code.trim).mkString("\n")
    +    variables.foreach(_.code = "")
    +    evaluate
    +  }
    +
    +  /**
    +    * Returns source code to evaluate the variables for required attributes, and clear the code
    +    * of evaluated variables, to prevent them to be evaluated twice..
         */
    +  protected def evaluateRequiredVariables(
    +      attributes: Seq[Attribute],
    +      variables: Seq[ExprCode],
    +      required: AttributeSet): String = {
    +    var evaluateVars = ""
    +    variables.zipWithIndex.foreach { case (ev, i) =>
    +      if (ev.code != "" && required.contains(attributes(i))) {
    --- End diff --
    
    @davies I was just reviewing build warnings, and it flags this line. `ev.code` is a `Block` rather than `String`. Should it be `ev.code.nonEmpty && ...` instead?


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org