You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by "1427357147@qq.com" <14...@qq.com> on 2018/03/26 03:36:15 UTC
the issue about the + in column,can we support the string please?
Hi all,
I have a table like below:
+---+---------+-----------+
| id| name|sharding_id|
+---+---------+-----------+
| 1|leader us| 10000|
| 3| mycat| 10000|
+---+---------+-----------+
My schema is :
root
|-- id: integer (nullable = false)
|-- name: string (nullable = true)
|-- sharding_id: integer (nullable = false)
I want add a new column named newName. The new column is based on "name" and append "abc" after it. My code looks like:
stud_scoreDF.withColumn("newName", stud_scoreDF.col("name") + "abc" ).show()
When I run the code, I got the reslult:
+---+---------+-----------+-------+
| id| name|sharding_id|newName|
+---+---------+-----------+-------+
| 1|leader us| 10000| null|
| 3| mycat| 10000| null|
+---+---------+-----------+-------+
I checked the code, the key code is in arithmetic.scala. line 165.
It looks like:
override def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode = dataType match {
case dt: DecimalType =>
defineCodeGen(ctx, ev, (eval1, eval2) => s"$eval1.$$plus($eval2)")
case ByteType | ShortType =>
defineCodeGen(ctx, ev,
(eval1, eval2) => s"(${ctx.javaType(dataType)})($eval1 $symbol $eval2)")
case CalendarIntervalType =>
defineCodeGen(ctx, ev, (eval1, eval2) => s"$eval1.add($eval2)")
case _ =>
defineCodeGen(ctx, ev, (eval1, eval2) => s"$eval1 $symbol $eval2")
}
My issue is:
Can we add case StringType in this class to support string append please?
1427357147@qq.com