You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "shenghuang (JIRA)" <ji...@apache.org> on 2019/05/23 10:40:00 UTC

[jira] [Created] (SPARK-27817) Is it possible to achieve global scheduling optimization by predicting task execution times (e.g., training models with historical data using machine learning)?

shenghuang created SPARK-27817:
----------------------------------

             Summary: Is it possible to achieve global scheduling optimization by predicting task execution times (e.g., training models with historical data using machine learning)?
                 Key: SPARK-27817
                 URL: https://issues.apache.org/jira/browse/SPARK-27817
             Project: Spark
          Issue Type: New Feature
          Components: Spark Core
    Affects Versions: 2.4.3
            Reporter: shenghuang


For example, by predicting the execution time of ANY level task, 3 seconds before the task is about to be completed,Copy the data for the next ANY task to the current node ahead of time.Thus reduces the total execution time of the application.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org