You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "wangyum (via GitHub)" <gi...@apache.org> on 2023/04/01 07:09:11 UTC
[GitHub] [spark] wangyum opened a new pull request, #40633: [SPARK-43000][SQL] Do not cast to double type in `PromoteStrings`
wangyum opened a new pull request, #40633:
URL: https://github.com/apache/spark/pull/40633
### What changes were proposed in this pull request?
This PR updates `PromoteStrings` to not cast to double type when one side is `AnsiIntervalType` in `BinaryArithmetic`.
### Why are the changes needed?
The error message is incorrect. For example:
```sql
select '2' / interval 2 second;
```
The error message is: the left and right operands of the binary operator have incompatible types ("DOUBLE" and "INTERVAL SECOND").
It should be: the left and right operands of the binary operator have incompatible types ("STRING" and "INTERVAL SECOND").
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Update existing unit test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] closed pull request #40633: [SPARK-43000][SQL] Do not cast to double type in `PromoteStrings`
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #40633: [SPARK-43000][SQL] Do not cast to double type in `PromoteStrings`
URL: https://github.com/apache/spark/pull/40633
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #40633: [SPARK-43000][SQL] Do not cast to double type in `PromoteStrings`
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on PR #40633:
URL: https://github.com/apache/spark/pull/40633#issuecomment-1493483913
cc @gengliangwang too
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on a diff in pull request #40633: [SPARK-43000][SQL] Do not cast to double type in `PromoteStrings`
Posted by "gengliangwang (via GitHub)" <gi...@apache.org>.
gengliangwang commented on code in PR #40633:
URL: https://github.com/apache/spark/pull/40633#discussion_r1156330211
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala:
##########
@@ -1107,10 +1107,12 @@ object TypeCoercion extends TypeCoercionBase {
case e if !e.childrenResolved => e
case a @ BinaryArithmetic(left @ StringType(), right)
- if right.dataType != CalendarIntervalType =>
+ if right.dataType != CalendarIntervalType &&
+ !right.dataType.isInstanceOf[AnsiIntervalType] =>
a.makeCopy(Array(Cast(left, DoubleType), right))
case a @ BinaryArithmetic(left, right @ StringType())
- if left.dataType != CalendarIntervalType =>
+ if left.dataType != CalendarIntervalType &&
+ !left.dataType.isInstanceOf[AnsiIntervalType] =>
Review Comment:
How about
`select INTERVAL '02' SECOND * '2'`
It used to work, but now it fails
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on pull request #40633: [SPARK-43000][SQL] Do not cast to double type in `PromoteStrings`
Posted by "wangyum (via GitHub)" <gi...@apache.org>.
wangyum commented on PR #40633:
URL: https://github.com/apache/spark/pull/40633#issuecomment-1493472727
cc @cloud-fan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] commented on pull request #40633: [SPARK-43000][SQL] Do not cast to double type in `PromoteStrings`
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #40633:
URL: https://github.com/apache/spark/pull/40633#issuecomment-1635085303
We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wangyum commented on a diff in pull request #40633: [SPARK-43000][SQL] Do not cast to double type in `PromoteStrings`
Posted by "wangyum (via GitHub)" <gi...@apache.org>.
wangyum commented on code in PR #40633:
URL: https://github.com/apache/spark/pull/40633#discussion_r1156658446
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala:
##########
@@ -1107,10 +1107,12 @@ object TypeCoercion extends TypeCoercionBase {
case e if !e.childrenResolved => e
case a @ BinaryArithmetic(left @ StringType(), right)
- if right.dataType != CalendarIntervalType =>
+ if right.dataType != CalendarIntervalType &&
+ !right.dataType.isInstanceOf[AnsiIntervalType] =>
a.makeCopy(Array(Cast(left, DoubleType), right))
case a @ BinaryArithmetic(left, right @ StringType())
- if left.dataType != CalendarIntervalType =>
+ if left.dataType != CalendarIntervalType &&
+ !left.dataType.isInstanceOf[AnsiIntervalType] =>
Review Comment:
It's already handled by `ImplicitTypeCasts`:
https://github.com/apache/spark/blob/2a38dfa6c9187417a998e7062d15d5e88e82bd14/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala#L704-L709
https://github.com/apache/spark/blob/51e4c2cc55aa01f07b28b1cd807b553f8729075d/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/intervalExpressions.scala#L562
https://github.com/apache/spark/blob/2a38dfa6c9187417a998e7062d15d5e88e82bd14/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala#L985-L988
In fact, we already have relevant tests and will not change the results of these queries:
https://github.com/apache/spark/blob/738a81e3173bd6571e038196fc161737ca105f58/sql/core/src/test/resources/sql-tests/inputs/interval.sql#L14-L31
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a diff in pull request #40633: [SPARK-43000][SQL] Do not cast to double type in `PromoteStrings`
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on code in PR #40633:
URL: https://github.com/apache/spark/pull/40633#discussion_r1156592711
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala:
##########
@@ -1107,10 +1107,12 @@ object TypeCoercion extends TypeCoercionBase {
case e if !e.childrenResolved => e
case a @ BinaryArithmetic(left @ StringType(), right)
- if right.dataType != CalendarIntervalType =>
+ if right.dataType != CalendarIntervalType &&
+ !right.dataType.isInstanceOf[AnsiIntervalType] =>
a.makeCopy(Array(Cast(left, DoubleType), right))
case a @ BinaryArithmetic(left, right @ StringType())
- if left.dataType != CalendarIntervalType =>
+ if left.dataType != CalendarIntervalType &&
+ !left.dataType.isInstanceOf[AnsiIntervalType] =>
Review Comment:
good point. For `interval * string`, there seems to be only one choice: the string should be a number.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org