[GitHub] [hudi] KnightChess commented on pull request #8794: [HUDI-6257] fix upgrade table version check partition path with hive …

["KnightChess (via GitHub)" <gi...@apache.org>]

KnightChess commented on PR #8794:
URL: https://github.com/apache/hudi/pull/8794#issuecomment-1573904820

   @nsivabalan ehhh, no, will success when upgrade table version from 4 to 5, but will throw type change error when scan the table in SparkHoodieTableFileIndex parse partition values if partition col type is not `String`, because `DEFAULT_PARTITION_PATH` type is  string. But if table partition col type is string, It will only be query.
   ```scala
   def castPartValueToDesiredType(
         desiredType: DataType,
         value: String,
         zoneId: ZoneId): Any = desiredType match {
       case _ if value == DEFAULT_PARTITION_PATH => null
       case NullType => null
       case BooleanType => JBoolean.parseBoolean(value)
       case StringType => UTF8String.fromString(unescapePathName(value))
       case IntegerType => Integer.parseInt(value)
       case LongType => JLong.parseLong(value)
       case DoubleType => JDouble.parseDouble(value)
       case _: DecimalType => Literal(new JBigDecimal(value)).value
       case DateType =>
         Cast(Literal(value), DateType, Some(zoneId.getId)).eval()
       // Timestamp types
       case dt: TimestampType =>
         Try {
           Cast(Literal(unescapePathName(value)), dt, Some(zoneId.getId)).eval()
         }.getOrElse {
           Cast(Cast(Literal(value), DateType, Some(zoneId.getId)), dt).eval()
         }
       case dt => throw new IllegalArgumentException(s"Unexpected type $dt")
     }
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org