[jira] [Commented] (SPARK-16759) Spark expose an API to pass in Caller Context into it

["Aihua Xu (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-16759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16679135#comment-16679135 ] 

Aihua Xu commented on SPARK-16759:
----------------------------------

Seems we should implement this callerContext in SparkContext rather than in SparkConfiguration since e.g., if I submit multiple hive jobs in the same session to Spark, then the spark application will have the same queryId. I'm wondering if we can reuse SparkContext.JobGroup as such callerContext or implement CallerContext within SparkContext so each new job will have unique value.

> Spark expose an API to pass in Caller Context into it
> -----------------------------------------------------
>
>                 Key: SPARK-16759
>                 URL: https://issues.apache.org/jira/browse/SPARK-16759
>             Project: Spark
>          Issue Type: Sub-task
>            Reporter: Weiqing Yang
>            Assignee: Weiqing Yang
>            Priority: Major
>             Fix For: 2.2.0
>
>
> The API should expose a way for the upstream components to inject a caller context. Caller context will in the form of a tuple ( caller context type, caller context id ). Initial implementation will require support of at least one primary caller context. Future versions may need secondary caller contexts to also be provided via the same interface.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org