You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "liupengcheng (Jira)" <ji...@apache.org> on 2020/03/10 09:41:00 UTC

[jira] [Created] (SPARK-31105) Respect sql execution id when scheduling taskSets

liupengcheng created SPARK-31105:
------------------------------------

             Summary: Respect sql execution id when scheduling taskSets
                 Key: SPARK-31105
                 URL: https://issues.apache.org/jira/browse/SPARK-31105
             Project: Spark
          Issue Type: Improvement
          Components: Spark Core
    Affects Versions: 3.0.0
            Reporter: liupengcheng


Currently, spark will sort taskSets by jobId and stageId and then schedule them in order for FIFO schedulingMode. In OLAP senerios, especially under high concurrency, the taskSets are always from different sql queries and several jobs can be submitted for execution at one time  for one query for adaptive execution. But now we order those taskSets without considering the execution group, which may causes the query being delayed.
So I propose to consider the sql execution id when scheduling jobs.




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org