[GitHub] [hudi] nsivabalan commented on a diff in pull request #7857: [HUDI-5704] De-coupling column drop flag and schema validation flag

["nsivabalan (via GitHub)" <gi...@apache.org>]

nsivabalan commented on code in PR #7857:
URL: https://github.com/apache/hudi/pull/7857#discussion_r1096633834


##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala:
##########
@@ -455,7 +455,22 @@ object HoodieSparkSqlWriter {
           //       w/ the table's one and allow schemas to diverge. This is required in cases where
           //       partial updates will be performed (for ex, `MERGE INTO` Spark SQL statement) and as such
           //       only incoming dataset's projection has to match the table's schema, and not the whole one
-          if (!shouldValidateSchemasCompatibility || isSchemaCompatible(latestTableSchema, canonicalizedSourceSchema, allowAutoEvolutionColumnDrop)) {
+
+          if (!shouldValidateSchemasCompatibility) {
+            // if no validation is enabled, check for col drop
+            // if col drop is allowed, go ahead. if not, check for projection, so that we do not allow dropping cols
+            if (allowAutoEvolutionColumnDrop || checkProjection(latestTableSchema, canonicalizedSourceSchema)) {
+              canonicalizedSourceSchema

Review Comment:
   yes, just that this is only incase when schema validation is not enabled. If enabled, we follow previous logic. you can find it below. 
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org