[jira] [Created] (SPARK-45205) Since version 3.2.0, Spark SQL has taken longer to execute "show paritions",probably because of changes introduced by SPARK-35278

["Qiang Yang (Jira)" <ji...@apache.org>]

Qiang Yang created SPARK-45205:
----------------------------------

             Summary: Since version 3.2.0, Spark SQL has taken longer to execute "show paritions",probably because of changes introduced by  SPARK-35278
                 Key: SPARK-45205
                 URL: https://issues.apache.org/jira/browse/SPARK-45205
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 3.2.0
            Reporter: Qiang Yang


After SPARK-35378 was changed, I noticed that the execution of statements such as ‘show parititions test' became slower.

The change point is that the execution process changes from ExecutedCommandEnec to CommandResultExec, but ExecutedCommandExec originally implemented the following method

override def executeToIterator(): Iterator[InternalRow] = sideEffectResult.iterator

CommandResultExec is not rewritten, so when the hasNext method is executed, a job process is created, resulting in increased time-consuming



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org