You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Apache Spark (Jira)" <ji...@apache.org> on 2021/09/03 04:08:00 UTC
[jira] [Assigned] (SPARK-36658) Expose executionId to
QueryExecutionListener
[ https://issues.apache.org/jira/browse/SPARK-36658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-36658:
------------------------------------
Assignee: Apache Spark
> Expose executionId to QueryExecutionListener
> --------------------------------------------
>
> Key: SPARK-36658
> URL: https://issues.apache.org/jira/browse/SPARK-36658
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.1.2
> Reporter: huangtengfei
> Assignee: Apache Spark
> Priority: Minor
>
> Now in [QueryExecutionListener|https://github.com/apache/spark/blob/v3.2.0-rc2/sql/core/src/main/scala/org/apache/spark/sql/util/QueryExecutionListener.scala#L38] we have exposed API to get the query execution information:
> def onSuccess(funcName: String, qe: QueryExecution, durationNs: Long): Unit
> def onFailure(funcName: String, qe: QueryExecution, exception: Exception): Unit
>
> But we can not get a clear information that which query is this. In Spark SQL, I think that executionId is the direct identifier of a query execution. So I think it make sense to expose executionId to the QueryExecutionListener, so that people can easily find the exact query in UI or history server to track more information of the query execution. And there is no easy way we can find the relevant executionId from a QueryExecution object.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org