You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Thomas Graves (Jira)" <ji...@apache.org> on 2020/01/07 13:59:00 UTC

[jira] [Created] (SPARK-30446) Accelerator aware scheduling checkResourcesPerTask doesn't cover all cases

Thomas Graves created SPARK-30446:
-------------------------------------

             Summary: Accelerator aware scheduling checkResourcesPerTask doesn't cover all cases
                 Key: SPARK-30446
                 URL: https://issues.apache.org/jira/browse/SPARK-30446
             Project: Spark
          Issue Type: Bug
          Components: Spark Core
    Affects Versions: 3.0.0
            Reporter: Thomas Graves


with accelerator aware scheduling SparkContext.checkResourcesPerTask

Tries to make sure that users have configured things properly and warn or error if not.

It doesn't properly handle all cases like warning if cpu resources are being wasted.  We should test this better and handle those. 

I fixed these in the stage level scheduling but not sure the timeline on getting that in so we may want to fix this separately as well.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org