You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "zzzzming95 (Jira)" <ji...@apache.org> on 2023/04/08 13:42:00 UTC

[jira] [Created] (SPARK-43074) Add the function without constant parameters of `SessionState#executePlan`

zzzzming95 created SPARK-43074:
----------------------------------

             Summary: Add the function without constant parameters of `SessionState#executePlan`
                 Key: SPARK-43074
                 URL: https://issues.apache.org/jira/browse/SPARK-43074
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 3.2.3
            Reporter: zzzzming95


```

df = spark.sql("select 1") 
catalyst_plan = df._jdf.queryExecution().logical()
print('catalyst_plan: ', catalyst_plan)
df_size = spark._jsparkSession.sessionState().executePlan(catalyst_plan)

```

 

will get exception beacause py4j not support default value of scala.

 

https://github.com/apache/spark/blob/d8b720a579a456aec9c7f843d7feaa1454ebf9d4/sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala#L124

```

def executePlan(
plan: LogicalPlan,
mode: CommandExecutionMode.Value = CommandExecutionMode.ALL): QueryExecution =
createQueryExecution(plan, mode)

```

 

Will get Exception:

 

```

py4j.protocol.Py4JError: An error occurred while calling o87.executePlan. Trace:
py4j.Py4JException: Method executePlan([class org.apache.spark.sql.catalyst.plans.logical.Project]) does not exist
    at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:321)
    at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:329)
    at py4j.Gateway.invoke(Gateway.java:274)
    at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
    at py4j.commands.CallCommand.execute(CallCommand.java:79)
    at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)
    at py4j.ClientServerConnection.run(ClientServerConnection.java:106)
    at java.lang.Thread.run(Thread.java:748)

```



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org