[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31133: [SPARK-26836][SQL] Supporting Avro schema evolution for partitioned Hive tables

[GitBox <gi...@apache.org>]

dongjoon-hyun commented on a change in pull request #31133:
URL: https://github.com/apache/spark/pull/31133#discussion_r568424773



##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala
##########
@@ -388,6 +394,9 @@ private[hive] object HiveTableUtil {
 private[hive] object DeserializerLock
 
 private[hive] object HadoopTableReader extends HiveInspectors with Logging {
+
+  val avroTableProperties = AvroTableProperties.values().map(_.getPropName()).toSet

Review comment:
       SPARK-26836 is filed for 2.3.1 ~ 3.2.0 and this patch depends on `org.apache.hadoop.hive.serde2.avro.AvroSerdeUtils.AvroTableProperties`. I'm wondering if this patch is applicable with old Spark branches with old Hive libraries. For example,
   - Apache Spark master : 2.3.8
   - Apache Spark branch-3.1: 2.3.7
   - Apache Spark branch-3.0: 2.3.7 and 1.2.1.spark2
   - Apache Spark branch-2.4: 1.2.1.spark2
   
   What is the minimum version of Hive library to use this? Is there a known bug for this `AvroTableProperties` classes?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org