You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "xie shuiahu (Jira)" <ji...@apache.org> on 2024/10/12 09:09:00 UTC

[jira] [Created] (SPARK-49940) make "limit n" in spark connect use executeCollect

xie shuiahu created SPARK-49940:
-----------------------------------

             Summary: make "limit n" in spark connect use executeCollect
                 Key: SPARK-49940
                 URL: https://issues.apache.org/jira/browse/SPARK-49940
             Project: Spark
          Issue Type: Improvement
          Components: Connect
    Affects Versions: 3.5.3
            Reporter: xie shuiahu


```SQL

spark.sql("select * from some_tables limit 10").collect()

```

In spark connect, this sql will call `SparkPlan.execute` and run all partitions. But in spark it will call `executeCollect` which is more performant for `CollectLimitExec` and `CollectTailExec`



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org