Re: [I] Type mismatch issue when Spark writes to Hudi and synchronizes metadata [hudi]

["nsivabalan (via GitHub)" <gi...@apache.org>]

nsivabalan commented on issue #16327:
URL: https://github.com/apache/hudi/issues/16327#issuecomment-4795581112

   This appears to be fixed on current `master`. The catalog/Hudi schema mismatch behind `Could not convert field Type from TINYINT to int` was addressed by [HUDI-6219](https://issues.apache.org/jira/browse/HUDI-6219) (commit `ea547e5681a0`, PR #8725, May 2023): `HoodieCatalogTable.tableSchema` now loads the schema from the Hudi meta directory first, and `CreateHoodieTableCommand` writes that canonical schema into the Spark catalog — so byte/short columns (which Avro/Hudi normalize to INT in `HoodieSparkSchemaConverters.scala`) are stored as INT on both sides, eliminating the mismatch on read. A regression test using `id tinyint` exists at `TestCreateTable.scala` ("Test hoodie table schema consistency for non-Avro data types"). The originally linked PR #10229 was closed without merging, presumably because HUDI-6219 already covered it. Closing as fixed unless you can still reproduce on a recent release — please reopen with the Hudi/Spark versions if so.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org