Re: [I] Spark reports an error when modifying the schema of the Flink table [hudi]

["nsivabalan (via GitHub)" <gi...@apache.org>]

nsivabalan commented on issue #15563:
URL: https://github.com/apache/hudi/issues/15563#issuecomment-4795445777

   This appears to be the same problem tracked as HUDI-5585 and fixed in PR [#7706](https://github.com/apache/hudi/pull/7706) (commit `9469882d80f6`), which is in current master. The fix adds `supplementMetaFields()` in `hudi-flink/.../catalog/TableOptionProperties.java` so that the Spark-compatible parquet schema metadata stored in HMS by the Flink Hudi catalog includes the `_hoodie_*` meta-columns — without those, Spark's `HiveExternalCatalog.alterTableDataSchema` path fails when trying to ALTER a Flink-created table.
   
   Could you retry on a recent master build (or 0.13.0+) and confirm whether you still hit this? If so, please share the table's HMS properties (especially `spark.sql.sources.schema.*`) so we can tell where the meta-column gap is coming from. Otherwise this can be closed as fixed.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org