[GitHub] spark pull request #19393: [SPARK-21644][SQL] LocalLimit.maxRows is defined ...

[viirya <gi...@git.apache.org>]

Github user viirya commented on a diff in the pull request:

    https://github.com/apache/spark/pull/19393#discussion_r142311845
  
    --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala ---
    @@ -296,13 +296,20 @@ object LimitPushDown extends Rule[LogicalPlan] {
         }
       }
     
    -  private def maybePushLimit(limitExp: Expression, plan: LogicalPlan): LogicalPlan = {
    -    (limitExp, plan.maxRows) match {
    -      case (IntegerLiteral(maxRow), Some(childMaxRows)) if maxRow < childMaxRows =>
    +  private def maybePushLocalLimit(limitExp: Expression, plan: LogicalPlan): LogicalPlan = {
    +    (limitExp, plan.maxRowsPerPartition) match {
    +      case (IntegerLiteral(newLimit), Some(childMaxRows)) if newLimit < childMaxRows =>
    +        // If the child has a cap on max rows per partition and the cap is smaller than
    +        // the new limit, put a new LocalLimit there.
    --- End diff --
    
    I think it is `the cap is larger than the new limit`?


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org