[GitHub] [spark] rshkv commented on a diff in pull request #40902: [SPARK-43142] Fix DSL expressions on attributes with special characters

["rshkv (via GitHub)" <gi...@apache.org>]

rshkv commented on code in PR #40902:
URL: https://github.com/apache/spark/pull/40902#discussion_r1174398567


##########
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionSQLBuilderSuite.scala:
##########
@@ -95,7 +95,7 @@ class ExpressionSQLBuilderSuite extends SparkFunSuite {
 
   test("attributes") {
     checkSQL($"a".int, "a")
-    checkSQL(Symbol("foo bar").int, "`foo bar`")
+    checkSQL(Symbol("`foo bar`").int, "`foo bar`")

Review Comment:
   `Symbol("foo bar").int` previously just passed `"foo bar"` to the `AttributeReference` without parsing an `UnresolvedAttribute`. We now call `attr()` which _is_ parsing `"foo bar"` and that fails without backticks.
   
   Alternatively, we can `quoteIfNeeded` in `DslSymbol` and treat, e.g., `Symbol("foo.bar")` as single-part (which is the current behaviour). But I think it's confusing if `$"foo.bar"` returns a multi-part attribute and `Symbol("foo.bar")` doesn't.



##########
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionSQLBuilderSuite.scala:
##########
@@ -95,7 +95,7 @@ class ExpressionSQLBuilderSuite extends SparkFunSuite {
 
   test("attributes") {
     checkSQL($"a".int, "a")
-    checkSQL(Symbol("foo bar").int, "`foo bar`")
+    checkSQL(Symbol("`foo bar`").int, "`foo bar`")

Review Comment:
   `Symbol("foo bar").int` previously just passed `"foo bar"` to the `AttributeReference` without parsing an `UnresolvedAttribute`. We now call `attr()` which _is_ parsing `"foo bar"` and that fails without backticks.
   
   Alternatively, we can `quoteIfNeeded` in `DslSymbol` and treat, e.g., `Symbol("foo.bar")` as single-part (which is the current behaviour). But I think it's confusing if `$"foo.bar"` returns a multi-part attribute and `Symbol("foo.bar")` doesn't.
   
   So I rather just update the test.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org