You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "hudi-bot (via GitHub)" <gi...@apache.org> on 2025/11/30 04:46:10 UTC
[I] Support Tinyint and small int data types [hudi]
hudi-bot opened a new issue, #15056:
URL: https://github.com/apache/hudi/issues/15056
Currently, the avro schema does not support tinyint and smallint datatypes, but Hudi uses the avro schema as bridge of user DDL schema and parquet schema, we should fix that.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-3608
- Type: Task
---
## Comments
29/Mar/23 08:58;weiming;[~danny0405] We have encountered the same problem, when is this bug expected to be fixed?;;;
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Support Tinyint and small int data types [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan closed issue #15056: Support Tinyint and small int data types
URL: https://github.com/apache/hudi/issues/15056
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Support Tinyint and small int data types [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on issue #15056:
URL: https://github.com/apache/hudi/issues/15056#issuecomment-4795307299
Triage update: this appears to be resolved on current `master`.
- Spark path: `ByteType`/`ShortType` are mapped to Avro `INT` in [`HoodieSparkSchemaConverters.scala`](https://github.com/apache/hudi/blob/master/hudi-client/hudi-spark-client/src/main/scala/org/apache/spark/sql/avro/HoodieSparkSchemaConverters.scala#L123) — the same widening Spark's own `spark-avro` does, since Avro has no native tinyint/smallint type.
- Tests cover this end-to-end:
- [`TestCreateTable.scala`](https://github.com/apache/hudi/blob/master/hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/ddl/TestCreateTable.scala#L1559) — *Test hoodie table schema consistency for non-Avro data types* creates a Hudi table with a `tinyint` primary key.
- [`TestHoodieSchemaConversionUtils.scala`](https://github.com/apache/hudi/blob/master/hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/TestHoodieSchemaConversionUtils.scala#L37) — round-trips byte/short fields.
- Flink path has explicit TINYINT/SMALLINT handling in `ImplicitTypeConverter`, `DataTypeUtils`, and `ParquetSplitReaderUtil`.
If you're still seeing a concrete failure (e.g. a specific DDL/query that errors out on a tinyint/smallint column), please share the engine version, the SQL, and the stack trace and we can reopen with a focused repro. Otherwise, suggest closing this and the linked HUDI-3608.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org