You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by ryanbald <gi...@git.apache.org> on 2017/08/24 06:25:28 UTC
[GitHub] spark pull request #19033: [SPARK-21811][SQL]Inconsistency when finding the ...
GitHub user ryanbald opened a pull request:
https://github.com/apache/spark/pull/19033
[SPARK-21811][SQL]Inconsistency when finding the widest common type of a combination of DateType, StringType, and NumericType
## What changes were proposed in this pull request?
When finding the widest common type of a sequence of data types where the beginning of the sequence consists solely of DateType/Timestamp and Numeric types and the following data type in the sequence is a StringType, the widest common type becomes StringType (which was not the case prior, as an AnalysisException was raised).
## How was this patch tested?
Added one test of Coalesce type coercion with the specific sequence of data types mentioned above and added onto a test of Union type coercion with a similar sequence of data types.
_Please state that the contribution is your original work and that you license the work to the project under the project’s open source license._
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ryanbald/spark type-coercion-bug
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/19033.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #19033
----
commit 9cdd9a653f909c766fde3b6c6b6f47f0684b1825
Author: Ryan Bald <ry...@levyx.com>
Date: 2017-08-24T05:43:14Z
when a combination of Date/Timestamp, Numeric, and String types are in a sequence of data types, the widest type is String
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #19033: [SPARK-21811][SQL]Inconsistency when finding the ...
Posted by jiangxb1987 <gi...@git.apache.org>.
Github user jiangxb1987 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19033#discussion_r170547020
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala ---
@@ -150,9 +150,27 @@ object TypeCoercion {
}
private def findWiderCommonType(types: Seq[DataType]): Option[DataType] = {
+ var awaitingString = false
--- End diff --
Maybe we should make a pass of all dataTypes here, and see whether there exists `StringType` and all other dataTypes can be promoted to `StringType`, under that case we can default return `StringType` instead of `None`, WDYT?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19033: [SPARK-21811][SQL]Inconsistency when finding the widest ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19033
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19033: [SPARK-21811][SQL]Inconsistency when finding the widest ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19033
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19033: [SPARK-21811][SQL]Inconsistency when finding the widest ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19033
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19033: [SPARK-21811][SQL]Inconsistency when finding the widest ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19033
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #19033: [SPARK-21811][SQL]Inconsistency when finding the ...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/19033
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org