You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by gatorsmile <gi...@git.apache.org> on 2018/12/07 08:07:38 UTC

[GitHub] spark pull request #22104: [SPARK-24721][SQL] Extract Python UDFs at the end...

Github user gatorsmile commented on a diff in the pull request:

    https://github.com/apache/spark/pull/22104#discussion_r239722680
  
    --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkOptimizer.scala ---
    @@ -31,7 +31,8 @@ class SparkOptimizer(
     
       override def defaultBatches: Seq[Batch] = (preOptimizationBatches ++ super.defaultBatches :+
         Batch("Optimize Metadata Only Query", Once, OptimizeMetadataOnlyQuery(catalog)) :+
    -    Batch("Extract Python UDF from Aggregate", Once, ExtractPythonUDFFromAggregate) :+
    +    Batch("Extract Python UDFs", Once,
    +      Seq(ExtractPythonUDFFromAggregate, ExtractPythonUDFs): _*) :+
    --- End diff --
    
    It looks weird to add this rule in our optimizer batch. We need at least some comments to explain the reason in the code. 


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org