[GitHub] spark pull request #19602: [SPARK-22384][SQL] Refine partition pruning when ...

[cloud-fan <gi...@git.apache.org>]

Github user cloud-fan commented on a diff in the pull request:

    https://github.com/apache/spark/pull/19602#discussion_r190312153
  
    --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala ---
    @@ -657,18 +656,39 @@ private[client] class Shim_v0_13 extends Shim_v0_12 {
     
         val useAdvanced = SQLConf.get.advancedPartitionPredicatePushdownEnabled
     
    +    object ExtractAttribute {
    +      def unapply(expr: Expression): Option[Attribute] = {
    +        expr match {
    +          case attr: Attribute => Some(attr)
    +          case cast @ Cast(child, dt: IntegralType, _) if child.dataType == StringType =>
    --- End diff --
    
    can we generalize it? I think the hive analyzer can also do type coercion so any safe cast should be allowed, like int -> long, int -> string, etc.
    
    We also need a config to safe guard it.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org