[jira] [Created] (SPARK-26746) Adaptive causes non-action operations to trigger computation

["eaton (JIRA)" <ji...@apache.org>]

eaton created SPARK-26746:
-----------------------------

             Summary: Adaptive causes non-action operations to trigger computation
                 Key: SPARK-26746
                 URL: https://issues.apache.org/jira/browse/SPARK-26746
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 2.4.0, 2.3.2, 2.3.1
            Reporter: eaton


When we turn on the spark. sql. adaptive. enabled switch, the following actions trigger the shuffle calculation, but not when the switch is off:

sql("select a, sum(a) from test group by a").rdd

The reason is _'_ExchangeCoordinator' submitMapStage too early, the code is like this:

while (j < submittedStageFutures.length) {
 // This call is a blocking call. If the stage has not finished, we will wait at here.
 mapOutputStatistics(j) = submittedStageFutures(j).get()
 j += 1
}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org