[jira] [Created] (SPARK-3626) Replace AsyncRDDActions with a more general async. API

["Josh Rosen (JIRA)" <ji...@apache.org>]

Josh Rosen created SPARK-3626:
---------------------------------

             Summary: Replace AsyncRDDActions with a more general async. API
                 Key: SPARK-3626
                 URL: https://issues.apache.org/jira/browse/SPARK-3626
             Project: Spark
          Issue Type: Improvement
          Components: Spark Core
            Reporter: Josh Rosen
            Assignee: Josh Rosen


The experimental AsyncRDDActions APIs seem to only exist in order to enable job cancellation.

We've been considering extending these APIs to support progress monitoring, but this would require stabilizing them so they're no longer {{@Experimental}}.

Instead, I propose to replace all of the AsyncRDDActions with a mechanism based on job groups which allows arbitrary computations to be run in job groups and supports cancellation / monitoring of Spark jobs launched from those computations.

(full design pending; see my GitHub PR for more details).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org