[GitHub] spark pull request #19389: [SPARK-22165][SQL] Resolve type conflicts between...

[cloud-fan <gi...@git.apache.org>]

Github user cloud-fan commented on a diff in the pull request:

    https://github.com/apache/spark/pull/19389#discussion_r150426911
  
    --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala ---
    @@ -468,14 +460,16 @@ object PartitioningUtils {
       }
     
       /**
    -   * Given a collection of [[Literal]]s, resolves possible type conflicts by up-casting "lower"
    -   * types.
    +   * Given a collection of [[Literal]]s, resolves possible type conflicts by
    +   * [[TypeCoercion.findWiderCommonType]]. See [[TypeCoercion.findWiderTypeForTwo]].
        */
       private def resolveTypeConflicts(literals: Seq[Literal], timeZone: TimeZone): Seq[Literal] = {
    -    val desiredType = {
    -      val topType = literals.map(_.dataType).maxBy(upCastingOrder.indexOf(_))
    --- End diff --
    
    I think it's a bug, the previous code didn't consider the case when input literal types are outside of the `upCastingOrder`, and just pick the first type as the final type.
    
    However I'm not sure what's the expected behavior. We need to figure out what's the possible data types for partition columns, and how to merge them.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org