You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by "Antonin Delpeuch (lists)" <li...@antonin.delpeuch.eu> on 2020/08/05 11:25:58 UTC

Async API to save RDDs?

Hi,

The RDD API provides async variants of a few RDD methods, which let the
user execute the corresponding jobs asynchronously. This makes it
possible to cancel the jobs for instance:
https://spark.apache.org/docs/latest/api/java/org/apache/spark/rdd/AsyncRDDActions.html

There does not seem to be async versions of the save methods such as
`saveAsTextFile`:
https://spark.apache.org/docs/latest/api/java/org/apache/spark/rdd/RDD.html#saveAsTextFile-java.lang.String-

Is there another way to start such jobs and get a handle on them (such
as the job id)? Specifically, I would like to be able to stop save jobs
on user request.

Thank you,
Antonin

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org