[jira] [Resolved] (SPARK-30795) Spark SQL codegen's code() interpolator should treat escapes like Scala's StringContext.s()

["Hyukjin Kwon (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-30795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hyukjin Kwon resolved SPARK-30795.
----------------------------------
    Fix Version/s: 3.1.0
       Resolution: Fixed

Fixed in https://github.com/apache/spark/pull/27544

> Spark SQL codegen's code() interpolator should treat escapes like Scala's StringContext.s()
> -------------------------------------------------------------------------------------------
>
>                 Key: SPARK-30795
>                 URL: https://issues.apache.org/jira/browse/SPARK-30795
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 2.4.0, 2.4.1, 2.4.2, 2.4.3, 2.4.4, 2.4.5, 3.0.0
>            Reporter: Kris Mok
>            Priority: Major
>             Fix For: 3.1.0
>
>
> The {{code()}} string interpolator in Spark SQL's code generator should treat escapes like Scala's builtin {{StringContext.s()}} interpolator, i.e. it should treat escapes in the code parts, and should not treat escapes in the input arguments.
> For example,
> {code}
> val arg = "This is an argument."
> val str = s"This is string part 1. $arg This is string part 2."
> val code = code"This is string part 1. $arg This is string part 2."
> assert(code.toString == str)
> {code}
> We should expect the {{code()}} interpolator produce the same thing as the {{StringContext.s()}} interpolator, where only escapes in the string parts should be treated, while the args should be kept verbatim.
> But in the current implementation, due to the eager folding of code parts and literal input args, the escape treatment is incorrectly done on both code parts and literal args.
> That causes a problem when an arg contains escape sequences and wants to preserve that in the final produced code string. For example, in {{Like}} expression's codegen, there's an ugly workaround for this bug:
> {code}
>       // We need double escape to avoid org.codehaus.commons.compiler.CompileException.
>       // '\\' will cause exception 'Single quote must be backslash-escaped in character literal'.
>       // '\"' will cause exception 'Line break in literal not allowed'.
>       val newEscapeChar = if (escapeChar == '\"' || escapeChar == '\\') {
>         s"""\\\\\\$escapeChar"""
>       } else {
>         escapeChar
>       }
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org