[jira] [Created] (SPARK-40475) Allow job status tracking with jobGroupId

["Anurag Mantripragada (Jira)" <ji...@apache.org>]

Anurag Mantripragada created SPARK-40475:
--------------------------------------------

             Summary: Allow job status tracking with jobGroupId
                 Key: SPARK-40475
                 URL: https://issues.apache.org/jira/browse/SPARK-40475
             Project: Spark
          Issue Type: Improvement
          Components: Spark Core, Web UI
    Affects Versions: 3.3.0
            Reporter: Anurag Mantripragada


Spark let's us group jobs together by setting a job group id. This is useful to check the job group in the web UI. For example

{{spark.sparkContext().setJobGroup("mygroup_id")}}

We have a use-case where we would like to have a long running Spark application and have jobs submitted to it. We would like to programmatically check the status of the jobs created by this group id. For example, [SQLStatusStore|#L41]] has `executionList()` which returns a map of jobs to the status. There is no way to filter this based on jobGroupId. 

This Jira is to add ability to get fine grained job statues by jobGroupId.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org