You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Liang Fenjie (Jira)" <ji...@apache.org> on 2022/08/27 07:10:00 UTC
[jira] [Updated] (SPARK-40242) Only return all physical plans after summitting pyspark script with several spark sql blocks inside
[ https://issues.apache.org/jira/browse/SPARK-40242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Liang Fenjie updated SPARK-40242:
---------------------------------
Description:
Backgroud:
In industry development environment, we got used to write several spark-sql blocks in one pyspark script. Without really submitting to cluster as running application, getting all the physical plans of a application is too hard to do within a parameter statement of "spark-submit" command.
Wish:
I wish to add a parameter "–genPlan" as for "spark-submit" command, which can return all the physical plans of a application instead of submitting to run indeed. Also, other approach to meet the issue is wellcome.
was:
Backgroud:
In industry development environment, we got used to write several spark-sql blocks in one pyspark script. Without really submitting to cluster as running application, getting all the physical plans of a application is too hard to do within a parameter statement of "spark-submit" command.
Wish:
I wish to add a parameter "–genPlan" as for "spark-submit" command, which can return all the physical plans of a application instead of submitting to run indeed.
> Only return all physical plans after summitting pyspark script with several spark sql blocks inside
> ---------------------------------------------------------------------------------------------------
>
> Key: SPARK-40242
> URL: https://issues.apache.org/jira/browse/SPARK-40242
> Project: Spark
> Issue Type: Wish
> Components: Spark Submit, SQL
> Affects Versions: 2.1.3
> Reporter: Liang Fenjie
> Priority: Major
>
> Backgroud:
> In industry development environment, we got used to write several spark-sql blocks in one pyspark script. Without really submitting to cluster as running application, getting all the physical plans of a application is too hard to do within a parameter statement of "spark-submit" command.
>
> Wish:
> I wish to add a parameter "–genPlan" as for "spark-submit" command, which can return all the physical plans of a application instead of submitting to run indeed. Also, other approach to meet the issue is wellcome.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org