You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "mihailom-db (via GitHub)" <gi...@apache.org> on 2024/03/28 10:00:20 UTC
[PR] [FOLLOWUP][SPARK-47431] Add session level default Collation [spark]
mihailom-db opened a new pull request, #45750:
URL: https://github.com/apache/spark/pull/45750
### What changes were proposed in this pull request?
Refactor of `literals.scala`.
### Why are the changes needed?
`Literal` expression is to low-level and should not access `SQLConf`.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Tests already available from initial PR.
### Was this patch authored or co-authored using generative AI tooling?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47431][SQL][FOLLOWUP] Do not access SQLConf in Literal.apply [spark]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan closed pull request #45750: [SPARK-47431][SQL][FOLLOWUP] Do not access SQLConf in Literal.apply
URL: https://github.com/apache/spark/pull/45750
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47431][FOLLOWUP] Do not access SQLConf in Literal.apply [spark]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on code in PR #45750:
URL: https://github.com/apache/spark/pull/45750#discussion_r1542955139
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/literals.scala:
##########
@@ -485,7 +484,7 @@ case class Literal (value: Any, dataType: DataType) extends LeafExpression {
override def sql: String = (value, dataType) match {
case (_, NullType | _: ArrayType | _: MapType | _: StructType) if value == null => "NULL"
case _ if value == null => s"CAST(NULL AS ${dataType.sql})"
- case (v: UTF8String, _: StringType) =>
Review Comment:
I don't think we should revert this change?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47431][SQL][FOLLOWUP] Do not access SQLConf in Literal.apply [spark]
Posted by "mihailom-db (via GitHub)" <gi...@apache.org>.
mihailom-db commented on code in PR #45750:
URL: https://github.com/apache/spark/pull/45750#discussion_r1544262650
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/literals.scala:
##########
@@ -485,9 +484,13 @@ case class Literal (value: Any, dataType: DataType) extends LeafExpression {
override def sql: String = (value, dataType) match {
case (_, NullType | _: ArrayType | _: MapType | _: StructType) if value == null => "NULL"
case _ if value == null => s"CAST(NULL AS ${dataType.sql})"
- case (v: UTF8String, _: StringType) =>
+ case (v: UTF8String, StringType) =>
// Escapes all backslashes and single quotes.
"'" + v.toString.replace("\\", "\\\\").replace("'", "\\'") + "'"
+ case (v: UTF8String, st: StringType) =>
+ // Escapes all backslashes and single quotes.
+ "'" + v.toString.replace("\\", "\\\\").replace("'", "\\'") +
+ "'" + st.typeName.substring(6)
Review Comment:
'stringName' COLLATE stringCollation
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47431][SQL][FOLLOWUP] Do not access SQLConf in Literal.apply [spark]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on PR #45750:
URL: https://github.com/apache/spark/pull/45750#issuecomment-2029295859
thanks, merging to master!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47431][SQL][FOLLOWUP] Do not access SQLConf in Literal.apply [spark]
Posted by "mihailom-db (via GitHub)" <gi...@apache.org>.
mihailom-db commented on code in PR #45750:
URL: https://github.com/apache/spark/pull/45750#discussion_r1544240476
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/literals.scala:
##########
@@ -485,7 +484,7 @@ case class Literal (value: Any, dataType: DataType) extends LeafExpression {
override def sql: String = (value, dataType) match {
case (_, NullType | _: ArrayType | _: MapType | _: StructType) if value == null => "NULL"
case _ if value == null => s"CAST(NULL AS ${dataType.sql})"
- case (v: UTF8String, _: StringType) =>
Review Comment:
Added a change for _: StringType, I would only say to wait until implicit casting goes in first, to resolve conflicts on this PR for some more tests that were added and that will be affected by this change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-47431][SQL][FOLLOWUP] Do not access SQLConf in Literal.apply [spark]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on code in PR #45750:
URL: https://github.com/apache/spark/pull/45750#discussion_r1544256831
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/literals.scala:
##########
@@ -485,9 +484,13 @@ case class Literal (value: Any, dataType: DataType) extends LeafExpression {
override def sql: String = (value, dataType) match {
case (_, NullType | _: ArrayType | _: MapType | _: StructType) if value == null => "NULL"
case _ if value == null => s"CAST(NULL AS ${dataType.sql})"
- case (v: UTF8String, _: StringType) =>
+ case (v: UTF8String, StringType) =>
// Escapes all backslashes and single quotes.
"'" + v.toString.replace("\\", "\\\\").replace("'", "\\'") + "'"
+ case (v: UTF8String, st: StringType) =>
+ // Escapes all backslashes and single quotes.
+ "'" + v.toString.replace("\\", "\\\\").replace("'", "\\'") +
+ "'" + st.typeName.substring(6)
Review Comment:
what does this produce?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org