You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (Jira)" <ji...@apache.org> on 2023/04/10 03:11:00 UTC
[jira] [Resolved] (SPARK-43074) Add the function without constant parameters of `SessionState#executePlan`
[ https://issues.apache.org/jira/browse/SPARK-43074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon resolved SPARK-43074.
----------------------------------
Resolution: Won't Fix
This isn't an API.
> Add the function without constant parameters of `SessionState#executePlan`
> --------------------------------------------------------------------------
>
> Key: SPARK-43074
> URL: https://issues.apache.org/jira/browse/SPARK-43074
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.2.3
> Reporter: zzzzming95
> Priority: Major
>
>
> {code:java}
> ```
> df = spark.sql("select 1")
> catalyst_plan = df._jdf.queryExecution().logical()
> print('catalyst_plan: ', catalyst_plan)
> df_size = spark._jsparkSession.sessionState().executePlan(catalyst_plan)
> ```
> {code}
>
>
> will get exception beacause py4j not support default value of scala.
>
> [https://github.com/apache/spark/blob/d8b720a579a456aec9c7f843d7feaa1454ebf9d4/sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala#L124]
>
> {code:java}
> ```
> def executePlan(
> plan: LogicalPlan,
> mode: CommandExecutionMode.Value = CommandExecutionMode.ALL): QueryExecution =
> createQueryExecution(plan, mode)
> ```
> {code}
>
>
> Will get Exception:
>
>
> {code:java}
> ```
> py4j.protocol.Py4JError: An error occurred while calling o87.executePlan. Trace:
> py4j.Py4JException: Method executePlan([class org.apache.spark.sql.catalyst.plans.logical.Project]) does not exist
> at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:321)
> at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:329)
> at py4j.Gateway.invoke(Gateway.java:274)
> at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
> at py4j.commands.CallCommand.execute(CallCommand.java:79)
> at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)
> at py4j.ClientServerConnection.run(ClientServerConnection.java:106)
> at java.lang.Thread.run(Thread.java:748)
> ```
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org