[jira] [Created] (SPARK-11203) UDF doesn't support charType column and lit function doesn't allow charType as argument

["M Bharat lal (JIRA)" <ji...@apache.org>]

M Bharat lal created SPARK-11203:
------------------------------------

             Summary: UDF doesn't support charType column and lit function doesn't allow charType as argument
                 Key: SPARK-11203
                 URL: https://issues.apache.org/jira/browse/SPARK-11203
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 1.5.0
            Reporter: M Bharat lal
            Priority: Minor


We have two issues

1) We cannot create dataframe with Char Type , see below example

scala> val employee = sqlContext.createDataFrame(Seq((1,"John"))).toDF("id","name")
employee: org.apache.spark.sql.DataFrame = [id: int, name: string]

scala> employee.withColumn("grade",lit('A'))
java.lang.RuntimeException: Unsupported literal type class java.lang.Character A
	at org.apache.spark.sql.catalyst.expressions.Literal$.apply(literals.scala:49)
	at org.apache.spark.sql.functions$.lit(functions.scala:89)


2) we have a function which takes string and char as input parameters and returns position of char in given string.  
 
registered function  as UDF and called the same UDF with characters literal which gave the below exception. Literal function doesn't support character as argument


scala> def strPos(x:String,c:Char):Integer = {x.indexOf(c)}
strPos: (x: String, c: Char)Integer

scala> sqlContext.udf.register("strPos",strPos _)
res13: org.apache.spark.sql.UserDefinedFunction = UserDefinedFunction(<function2>,IntegerType,List())

scala> df.select( callUDF("strPos",$"name",lit('J')))
java.lang.RuntimeException: Unsupported literal type class java.lang.Character J
	at org.apache.spark.sql.catalyst.expressions.Literal$.apply(literals.scala:49)

Can you please add this support or let us know if there is any other work around to achieve this




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org