You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sumeet Sharma (Jira)" <ji...@apache.org> on 2021/03/19 16:13:00 UTC
[jira] [Created] (SPARK-34804) registerFunction shouldnt be logging
warning message for same function being re-registered
Sumeet Sharma created SPARK-34804:
-------------------------------------
Summary: registerFunction shouldnt be logging warning message for same function being re-registered
Key: SPARK-34804
URL: https://issues.apache.org/jira/browse/SPARK-34804
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.1.1
Reporter: Sumeet Sharma
{code:java}
test("function registry warning") {
implicit val ss = spark
import ss.implicits._
val dd = Seq((1),(2)).toDF("a")
val dd1 = udf((i: Int) => i*2)
(1 to 4).foreach {
_ =>
dd.sparkSession.udf.register("function", dd1)
Thread.sleep(1000)
}
dd.withColumn("aa", expr("function(a)")).show(10)
}{code}
logs:
21/03/19 22:39:39 WARN SparkSession$Builder : Using an existing SparkSession; some spark core configurations may not take effect.
21/03/19 22:39:39 WARN SparkSession$Builder : Using an existing SparkSession; the static sql configurations will not take effect.
21/03/19 22:39:39 WARN SparkSession$Builder : Using an existing SparkSession; some spark core configurations may not take effect.
21/03/19 22:39:43 WARN SimpleFunctionRegistry : The function function replaced a previously registered function.
21/03/19 22:39:44 WARN SimpleFunctionRegistry : The function function replaced a previously registered function.
21/03/19 22:39:45 WARN SimpleFunctionRegistry : The function function replaced a previously registered function.
+---+---+
| a| aa|
+---+---+
| 1| 2|
| 2| 4|
+---+---+
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org