You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (Jira)" <ji...@apache.org> on 2022/08/11 03:41:00 UTC
[jira] [Updated] (SPARK-40026) spark-shell can not instantiate object whose class is defined in REPL dynamically
[ https://issues.apache.org/jira/browse/SPARK-40026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon updated SPARK-40026:
---------------------------------
Issue Type: Bug (was: Question)
> spark-shell can not instantiate object whose class is defined in REPL dynamically
> ---------------------------------------------------------------------------------
>
> Key: SPARK-40026
> URL: https://issues.apache.org/jira/browse/SPARK-40026
> Project: Spark
> Issue Type: Bug
> Components: Spark Shell
> Affects Versions: 2.4.8, 3.0.3
> Environment: Spark2.3.x ~ Spark3.0.x
> Scala2.11.x ~ Scala2.13.x
> Java 8
> Reporter: Kernel Force
> Priority: Major
>
> spark-shell throws {{NoSuchMethodException}} if I define a class in REPL and then call {{newInstance}} via reflection.
> {code:scala}
> Spark context available as 'sc' (master = yarn, app id = application_1656488084960_0162).
> Spark session available as 'spark'.
> Welcome to
> ____ __
> / __/__ ___ _____/ /__
> _\ \/ _ \/ _ `/ __/ '_/
> /___/ .__/\_,_/_/ /_/\_\ version 3.0.3
> /_/
>
> Using Scala version 2.12.10 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_141)
> Type in expressions to have them evaluated.
> Type :help for more information.
> scala> class Demo {
> | def demo(s: String): Unit = println(s)
> | }
> defined class Demo
> scala> classOf[Demo].newInstance().demo("OK")
> java.lang.InstantiationException: Demo
> at java.lang.Class.newInstance(Class.java:427)
> ... 47 elided
> Caused by: java.lang.NoSuchMethodException: Demo.<init>()
> at java.lang.Class.getConstructor0(Class.java:3082)
> at java.lang.Class.newInstance(Class.java:412)
> ... 47 more
> {code}
> But the same code works fine in native scala REPL:
> {code:scala}
> Welcome to Scala 2.12.10 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_131).
> Type in expressions for evaluation. Or try :help.
> scala> class Demo {
> | def demo(s: String): Unit = println(s)
> | }
> defined class Demo
> scala> classOf[Demo].newInstance().demo("OK")
> OK
> {code}
> What's the difference between spark-shell REPL and native scala REPL?
> I guess the Demo class might be treated as inner class in spark-shell REPL.
> But ... how to solve the problem?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org