You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "zeruibao (via GitHub)" <gi...@apache.org> on 2023/08/15 18:18:23 UTC

[GitHub] [spark] zeruibao opened a new pull request, #42503: [SPARK-43380][SQL] Fix Avro data type conversion issues without causing performance regression

zeruibao opened a new pull request, #42503:
URL: https://github.com/apache/spark/pull/42503

   ### What changes were proposed in this pull request?
   My last PR https://github.com/apache/spark/pull/41052 causes AVRO read performance regression since I change the code structure. I turn one match case into a nested match case. So I fix the Avro data type conversion issues in anther way to avoid this regression.
   
   Original Change: 
   We introduce the SQLConf `spark.sql.legacy.avro.allowReadingWithIncompatibleSchema` to prevent reading interval types as date or timestamp types to avoid getting corrupt dates as well as reading decimal types with incorrect precision.
   
   ### Why are the changes needed?
   We found the following issues with open source Avro:
   
   - Interval types can be read as date or timestamp types that would lead to wildly different results
      For example, `Duration.ofDays(1).plusSeconds(1)` will be read as `1972-09-27`, which is weird.
   - Decimal types can be read with lower precision, that leads to data being read as `null` instead of suggesting that a wider decimal format should be provided
   
   ### Does this PR introduce _any_ user-facing change?
   No
   
   ### How was this patch tested?
   Old unit test


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a diff in pull request #42503: [SPARK-43380][SQL] Fix Avro data type conversion issues without causing performance regression

Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on code in PR #42503:
URL: https://github.com/apache/spark/pull/42503#discussion_r1336523614


##########
common/utils/src/main/resources/error/error-classes.json:
##########
@@ -75,6 +75,16 @@
       }
     }
   },
+  "AVRO_INCORRECT_TYPE" : {

Review Comment:
   shall we unify the error class? `AVRO_INCOMPATIBLE_READ_TYPE`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a diff in pull request #42503: [SPARK-43380][SQL] Fix Avro data type conversion issues without causing performance regression

Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on code in PR #42503:
URL: https://github.com/apache/spark/pull/42503#discussion_r1338266812


##########
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala:
##########
@@ -4341,6 +4341,18 @@ object SQLConf {
       .booleanConf
       .createWithDefault(false)
 
+  val LEGACY_AVRO_ALLOW_INCOMPATIBLE_SCHEMA =
+    buildConf("spark.sql.legacy.avro.allowIncompatibleSchema")
+      .internal()
+      .doc("When set to false, if types in Avro are encoded in the same format, but " +
+        "the type in the Avro schema explicitly says that the data types are different, " +
+        "reject reading the data type in the format to avoid returning incorrect results. " +
+        "When set to true, it restores the legacy behavior of allow reading the data in the" +
+        " format, which may return incorrect results.")
+      .version("3.5.0")

Review Comment:
   ```suggestion
         .version("3.5.1")
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a diff in pull request #42503: [SPARK-43380][SQL] Fix Avro data type conversion issues without causing performance regression

Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on code in PR #42503:
URL: https://github.com/apache/spark/pull/42503#discussion_r1337071386


##########
common/utils/src/main/resources/error/error-classes.json:
##########
@@ -75,6 +75,16 @@
       }
     }
   },
+  "AVRO_INCOMPATIBLE_READ_TYPE" : {
+    "message" : [
+      "Cannot convert Avro <avroPath> to SQL <sqlPath> because the original encoded data type is <avroType>, however you're trying to read the field as <sqlType>, which would lead to an incorrect answer. To allow reading this field, enable the SQL configuration: \"spark.sql.legacy.avro.allowIncompatibleSchema\"."
+    ]
+  },
+  "AVRO_LOWER_PRECISION" : {

Review Comment:
   isn't it also `AVRO_INCOMPATIBLE_READ_TYPE`?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a diff in pull request #42503: [SPARK-43380][SQL] Fix Avro data type conversion issues without causing performance regression

Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on code in PR #42503:
URL: https://github.com/apache/spark/pull/42503#discussion_r1297941461


##########
connector/avro/src/main/scala/org/apache/spark/sql/avro/AvroDeserializer.scala:
##########
@@ -128,6 +133,36 @@ private[sql] class AvroDeserializer(
       case (INT, IntegerType) => (updater, ordinal, value) =>
         updater.setInt(ordinal, value.asInstanceOf[Int])
 
+      case (LONG, dt: TimestampType)

Review Comment:
   otherwise it's very hard to review the logic if these random patter match cases.



##########
connector/avro/src/main/scala/org/apache/spark/sql/avro/AvroDeserializer.scala:
##########
@@ -128,6 +133,36 @@ private[sql] class AvroDeserializer(
       case (INT, IntegerType) => (updater, ordinal, value) =>
         updater.setInt(ordinal, value.asInstanceOf[Int])
 
+      case (LONG, dt: TimestampType)

Review Comment:
   otherwise it's very hard to review the logic of these random patter match cases.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] zeruibao commented on a diff in pull request #42503: [SPARK-43380][SQL] Fix Avro data type conversion issues without causing performance regression

Posted by "zeruibao (via GitHub)" <gi...@apache.org>.
zeruibao commented on code in PR #42503:
URL: https://github.com/apache/spark/pull/42503#discussion_r1337471365


##########
common/utils/src/main/resources/error/error-classes.json:
##########
@@ -75,6 +75,16 @@
       }
     }
   },
+  "AVRO_INCOMPATIBLE_READ_TYPE" : {
+    "message" : [
+      "Cannot convert Avro <avroPath> to SQL <sqlPath> because the original encoded data type is <avroType>, however you're trying to read the field as <sqlType>, which would lead to an incorrect answer. To allow reading this field, enable the SQL configuration: \"spark.sql.legacy.avro.allowIncompatibleSchema\"."
+    ]
+  },
+  "AVRO_LOWER_PRECISION" : {

Review Comment:
   I still feel like these two error type should be separate. For `AVRO_LOWER_PRECISION`, we should just provide a wider decimal type. For `AVRO_INCOMPATIBLE_READ_TYPE`, we should provide a correct read type. It's different behavior.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] zeruibao commented on pull request #42503: [SPARK-43380][SQL] Fix Avro data type conversion issues without causing performance regression

Posted by "zeruibao (via GitHub)" <gi...@apache.org>.
zeruibao commented on PR #42503:
URL: https://github.com/apache/spark/pull/42503#issuecomment-1736770421

   https://github.com/zeruibao/spark/actions/runs/6318085817/job/17164945857
   All tests pass. Should be safe to merge.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] zeruibao commented on pull request #42503: [SPARK-43380][SQL] Fix Avro data type conversion issues without causing performance regression

Posted by "zeruibao (via GitHub)" <gi...@apache.org>.
zeruibao commented on PR #42503:
URL: https://github.com/apache/spark/pull/42503#issuecomment-1680887708

   Please review the PR. @gengliangwang @cloud-fan  Thank you!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] gengliangwang commented on a diff in pull request #42503: [SPARK-43380][SQL] Fix Avro data type conversion issues without causing performance regression

Posted by "gengliangwang (via GitHub)" <gi...@apache.org>.
gengliangwang commented on code in PR #42503:
URL: https://github.com/apache/spark/pull/42503#discussion_r1302257921


##########
connector/avro/src/main/scala/org/apache/spark/sql/avro/AvroDeserializer.scala:
##########
@@ -128,6 +133,36 @@ private[sql] class AvroDeserializer(
       case (INT, IntegerType) => (updater, ordinal, value) =>
         updater.setInt(ordinal, value.asInstanceOf[Int])
 
+      case (LONG, dt: TimestampType)

Review Comment:
   TimestampType/TimestampNTZType/DateType are all of DatetimeType. We can reduce duplicated code



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan closed pull request #42503: [SPARK-43380][SQL] Fix Avro data type conversion issues without causing performance regression

Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan closed pull request #42503: [SPARK-43380][SQL] Fix Avro data type conversion issues without causing performance regression
URL: https://github.com/apache/spark/pull/42503


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] zeruibao commented on a diff in pull request #42503: [SPARK-43380][SQL] Fix Avro data type conversion issues without causing performance regression

Posted by "zeruibao (via GitHub)" <gi...@apache.org>.
zeruibao commented on code in PR #42503:
URL: https://github.com/apache/spark/pull/42503#discussion_r1302261323


##########
connector/avro/src/main/scala/org/apache/spark/sql/avro/AvroDeserializer.scala:
##########
@@ -128,6 +133,36 @@ private[sql] class AvroDeserializer(
       case (INT, IntegerType) => (updater, ordinal, value) =>
         updater.setInt(ordinal, value.asInstanceOf[Int])
 
+      case (LONG, dt: TimestampType)

Review Comment:
   oh, that's a good idea!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a diff in pull request #42503: [SPARK-43380][SQL] Fix Avro data type conversion issues without causing performance regression

Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on code in PR #42503:
URL: https://github.com/apache/spark/pull/42503#discussion_r1297941203


##########
connector/avro/src/main/scala/org/apache/spark/sql/avro/AvroDeserializer.scala:
##########
@@ -128,6 +133,36 @@ private[sql] class AvroDeserializer(
       case (INT, IntegerType) => (updater, ordinal, value) =>
         updater.setInt(ordinal, value.asInstanceOf[Int])
 
+      case (LONG, dt: TimestampType)

Review Comment:
   Can we group the cases with some rules? e.g.
   ```
   case (INT, ...)
   case (INT, ...)
   ...
   case (LONG, ...)
   case (LONG, ...)
   ...
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] zeruibao commented on a diff in pull request #42503: [SPARK-43380][SQL] Fix Avro data type conversion issues without causing performance regression

Posted by "zeruibao (via GitHub)" <gi...@apache.org>.
zeruibao commented on code in PR #42503:
URL: https://github.com/apache/spark/pull/42503#discussion_r1302256752


##########
connector/avro/src/main/scala/org/apache/spark/sql/avro/AvroDeserializer.scala:
##########
@@ -128,6 +133,36 @@ private[sql] class AvroDeserializer(
       case (INT, IntegerType) => (updater, ordinal, value) =>
         updater.setInt(ordinal, value.asInstanceOf[Int])
 
+      case (LONG, dt: TimestampType)

Review Comment:
   Is `DatetimeType` the same as `TimestampType`? Why using `DatetimeType` is better?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a diff in pull request #42503: [SPARK-43380][SQL] Fix Avro data type conversion issues without causing performance regression

Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on code in PR #42503:
URL: https://github.com/apache/spark/pull/42503#discussion_r1336524055


##########
common/utils/src/main/resources/error/error-classes.json:
##########
@@ -75,6 +75,16 @@
       }
     }
   },
+  "AVRO_INCORRECT_TYPE" : {

Review Comment:
   We can just say that the type is incompatible and users can set the legacy config to restore the legacy undefined behavior.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a diff in pull request #42503: [SPARK-43380][SQL] Fix Avro data type conversion issues without causing performance regression

Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on code in PR #42503:
URL: https://github.com/apache/spark/pull/42503#discussion_r1336524185


##########
common/utils/src/main/resources/error/error-classes.json:
##########
@@ -75,6 +75,16 @@
       }
     }
   },
+  "AVRO_INCORRECT_TYPE" : {
+    "message" : [
+      "Cannot convert Avro <avroPath> to SQL <sqlPath> because the original encoded data type is <avroType>, however you're trying to read the field as <sqlType>, which would lead to an incorrect answer. To allow reading this field, enable the SQL configuration: <key>."

Review Comment:
   nit: we can hardcode the config name in the error message.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] gengliangwang commented on a diff in pull request #42503: [SPARK-43380][SQL] Fix Avro data type conversion issues without causing performance regression

Posted by "gengliangwang (via GitHub)" <gi...@apache.org>.
gengliangwang commented on code in PR #42503:
URL: https://github.com/apache/spark/pull/42503#discussion_r1300619239


##########
connector/avro/src/main/scala/org/apache/spark/sql/avro/AvroDeserializer.scala:
##########
@@ -128,6 +133,36 @@ private[sql] class AvroDeserializer(
       case (INT, IntegerType) => (updater, ordinal, value) =>
         updater.setInt(ordinal, value.asInstanceOf[Int])
 
+      case (LONG, dt: TimestampType)

Review Comment:
   Shall we just match `case(Long, DatetimeType)`?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] zeruibao commented on a diff in pull request #42503: [SPARK-43380][SQL] Fix Avro data type conversion issues without causing performance regression

Posted by "zeruibao (via GitHub)" <gi...@apache.org>.
zeruibao commented on code in PR #42503:
URL: https://github.com/apache/spark/pull/42503#discussion_r1336526095


##########
common/utils/src/main/resources/error/error-classes.json:
##########
@@ -75,6 +75,16 @@
       }
     }
   },
+  "AVRO_INCORRECT_TYPE" : {

Review Comment:
   sure!



##########
common/utils/src/main/resources/error/error-classes.json:
##########
@@ -75,6 +75,16 @@
       }
     }
   },
+  "AVRO_INCORRECT_TYPE" : {
+    "message" : [
+      "Cannot convert Avro <avroPath> to SQL <sqlPath> because the original encoded data type is <avroType>, however you're trying to read the field as <sqlType>, which would lead to an incorrect answer. To allow reading this field, enable the SQL configuration: <key>."

Review Comment:
   sounds good to me!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] zeruibao commented on a diff in pull request #42503: [SPARK-43380][SQL] Fix Avro data type conversion issues without causing performance regression

Posted by "zeruibao (via GitHub)" <gi...@apache.org>.
zeruibao commented on code in PR #42503:
URL: https://github.com/apache/spark/pull/42503#discussion_r1337471365


##########
common/utils/src/main/resources/error/error-classes.json:
##########
@@ -75,6 +75,16 @@
       }
     }
   },
+  "AVRO_INCOMPATIBLE_READ_TYPE" : {
+    "message" : [
+      "Cannot convert Avro <avroPath> to SQL <sqlPath> because the original encoded data type is <avroType>, however you're trying to read the field as <sqlType>, which would lead to an incorrect answer. To allow reading this field, enable the SQL configuration: \"spark.sql.legacy.avro.allowIncompatibleSchema\"."
+    ]
+  },
+  "AVRO_LOWER_PRECISION" : {

Review Comment:
   I still feel like these two error type should be separate. For `AVRO_LOWER_PRECISION`, we should just provide a wider decimal type. For `AVRO_INCOMPATIBLE_READ_TYPE`, we should provide a correct read type. It's different behavior.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] zeruibao commented on a diff in pull request #42503: [SPARK-43380][SQL] Fix Avro data type conversion issues without causing performance regression

Posted by "zeruibao (via GitHub)" <gi...@apache.org>.
zeruibao commented on code in PR #42503:
URL: https://github.com/apache/spark/pull/42503#discussion_r1337475524


##########
common/utils/src/main/resources/error/error-classes.json:
##########
@@ -75,6 +75,16 @@
       }
     }
   },
+  "AVRO_INCOMPATIBLE_READ_TYPE" : {
+    "message" : [
+      "Cannot convert Avro <avroPath> to SQL <sqlPath> because the original encoded data type is <avroType>, however you're trying to read the field as <sqlType>, which would lead to an incorrect answer. To allow reading this field, enable the SQL configuration: \"spark.sql.legacy.avro.allowIncompatibleSchema\"."
+    ]
+  },
+  "AVRO_LOWER_PRECISION" : {

Review Comment:
   sounds good. Let me unify these two error type.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on pull request #42503: [SPARK-43380][SQL] Fix Avro data type conversion issues without causing performance regression

Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on PR #42503:
URL: https://github.com/apache/spark/pull/42503#issuecomment-1736960671

   The last commit only change the config version, no need to retest. Thanks, merging to master/3.5!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org