You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Eren Avsarogullari (Jira)" <ji...@apache.org> on 2022/02/15 23:05:00 UTC
[jira] [Updated] (SPARK-38222) Expose Node Description attribute in SQL Rest API
[ https://issues.apache.org/jira/browse/SPARK-38222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eren Avsarogullari updated SPARK-38222:
---------------------------------------
Summary: Expose Node Description attribute in SQL Rest API (was: Expose nodeDesc attribute in SQL Rest API)
> Expose Node Description attribute in SQL Rest API
> -------------------------------------------------
>
> Key: SPARK-38222
> URL: https://issues.apache.org/jira/browse/SPARK-38222
> Project: Spark
> Issue Type: Task
> Components: SQL
> Affects Versions: 3.2.0
> Reporter: Eren Avsarogullari
> Priority: Major
>
> Currently, SQL public Rest API does not expose node description and it is useful to have nodeDesc attribute at query level to have more details such as:
> {code:java}
> - Join Operators(BHJ, SMJ, SHJ) => when correlating join operator with join type and which leg is built for BHJ.
> - HashAggregate => aggregated keys and agg functions
> - List can be extended for other physical operators.{code}
> *Current Sample Json Result:*
> {code:java}
> {
> "nodeId" : 14,
> "nodeName" : "BroadcastHashJoin",
> "wholeStageCodegenId" : 3,
> "stageIds" : [ 5 ],
> "metrics" : [ {
> "name" : "number of output rows",
> "value" : {
> "amount" : "2"
> }
> }
> },
> ...
> {
> "nodeId" : 8,
> "nodeName" : "HashAggregate",
> "wholeStageCodegenId" : 4,
> "stageIds" : [ 8 ],
> "metrics" : [ {
> "name" : "spill size",
> "value" : {
> "amount" : "0.0"
> }
> }
> } {code}
> *New* {*}Sample Json Result{*}{*}:{*}
> {code:java}
> {
> "nodeId" : 14,
> "nodeName" : "BroadcastHashJoin",
> "nodeDesc" : "BroadcastHashJoin [id#4], [id#24], Inner, BuildLeft, false",
> "wholeStageCodegenId" : 3,
> "stageIds" : [ 5 ],
> "metrics" : [ {
> "name" : "number of output rows",
> "value" : {
> "amount" : "2"
> }
> }
> },
> ...
> {
> "nodeId" : 8,
> "nodeName" : "HashAggregate",
> "nodeDesc" : "HashAggregate(keys=[name#5, age#6, salary#18], functions=[avg(cast(age#6 as bigint)), avg(salary#18)])",
> "wholeStageCodegenId" : 4,
> "stageIds" : [ 8 ],
> "metrics" : [ {
> "name" : "spill size",
> "value" : {
> "amount" : "0.0"
> }
> }
> } {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org