You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "kbuci (via GitHub)" <gi...@apache.org> on 2026/04/16 02:53:23 UTC
[I] Add Flink support for Blob data type [hudi]
kbuci opened a new issue, #18507:
URL: https://github.com/apache/hudi/issues/18507
### Feature Description
Feature Description
(Will split into smaller tickets as we get agreement)
All Flink read/write related classes should handle HoodieSchema Blob type.
- Based on initial scan, it seems Flink ROW/RowType <-> Avro RECORD type handling might already cover reading/writing Blob fields from Flink. Do we need to add more specific handling for exposing the HoodieSchema blob logical type when creating Flink ROW/RowType? Like passing in `"symbols": ["INLINE", "OUT_OF_LINE"]` ?
- Check if we need to add more Integ tests for Flink SQL We should make sure we test with multiple versions of Flink 2.x and for both COW and MOR tables.
- Add functional/integration tests to ensure Flink can read blob data records written by Spark writes, and vice versa.
- Exploration/comparison with Ray based on RFC https://github.com/apache/hudi/blob/master/rfc/rfc-100/rfc-100.md - should we even support (HUDI) Flink with blob if users can integrate with (HUDI) Ray for low-latency use cases?
### User Experience
**How users will use this feature:**
- Configuration changes needed
- API changes
- Usage examples
### Hudi RFC Requirements
**RFC PR link:** (if applicable)
**Why RFC is/isn't needed:**
- Does this change public interfaces/APIs? (Yes/No)
- Does this change storage format? (Yes/No)
- Justification:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add Flink support for Blob data type [hudi]
Posted by "kbuci (via GitHub)" <gi...@apache.org>.
kbuci commented on issue #18507:
URL: https://github.com/apache/hudi/issues/18507#issuecomment-4678410539
Thanks, https://github.com/apache/hudi/pull/18958 will tag you both once I confirm CIs pass and I promote it from draft to PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add Flink support for Blob data type [hudi]
Posted by "kbuci (via GitHub)" <gi...@apache.org>.
kbuci commented on issue #18507:
URL: https://github.com/apache/hudi/issues/18507#issuecomment-4666022999
@cshuo I started working on an initial PR for OOL blob support for Flink write (will plan to raise it soon). I also was planning to work on getting OOL blob support for Flink read (since I'm personally interested in the implementation of having Flink batch/fetch requests the same way that Spark-side impl currently handles)
But I haven't yet fully scoped out though the work needed specifically for Lance (w.r.t inline Blob).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add Flink support for Blob data type [hudi]
Posted by "cshuo (via GitHub)" <gi...@apache.org>.
cshuo commented on issue #18507:
URL: https://github.com/apache/hudi/issues/18507#issuecomment-4658103793
@kbuci have you already started working on this yet, if not, I can take it over.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add Flink support for Blob data type [hudi]
Posted by "cshuo (via GitHub)" <gi...@apache.org>.
cshuo commented on issue #18507:
URL: https://github.com/apache/hudi/issues/18507#issuecomment-4666245468
> [@cshuo](https://github.com/cshuo) I started working on an initial PR for OOL blob support for Flink write (will plan to raise it soon). I also was planning to work on getting OOL blob support for Flink read (since I'm personally interested in the implementation of having Flink batch/fetch requests the same way that Spark-side impl currently handles) But I haven't yet fully scoped out though the work needed specifically for Lance (w.r.t inline Blob).
Sg, thanks for the update! Feel free to ping me once your initial PR is up — happy to review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add Flink support for Blob data type [hudi]
Posted by "kbuci (via GitHub)" <gi...@apache.org>.
kbuci commented on issue #18507:
URL: https://github.com/apache/hudi/issues/18507#issuecomment-4464230282
I'll take this up
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add Flink support for Blob data type [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on issue #18507:
URL: https://github.com/apache/hudi/issues/18507#issuecomment-4666262229
@kbuci , thanks for the contribution~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org