You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@fluss.apache.org by "gyang94 (via GitHub)" <gi...@apache.org> on 2026/09/01 13:02:40 UTC

[PR] [kafka] Add schema-aware Produce transcoding [fluss]

gyang94 opened a new pull request, #4189:
URL: https://github.com/apache/fluss/pull/4189

   <!-- Generated-by: Codex (GPT-5.6) following https://github.com/apache/fluss/blob/main/AGENTS.md -->
   
   ### Purpose
   
   Linked issue: #4185
   
   Depends on #4188.
   
   This is PR 4 of 4 for the Basic Produce compatibility milestone. The intended review delta is the single tip commit `dc276fe8d`; the earlier stack commits remain visible until their PRs are merged and this branch is rebased.
   
   ### Brief change log
   
   - Resolve a Kafka record contract from Fluss table metadata.
   - Add raw, string, and JSON Kafka field decoders.
   - Support nested JSON projection, unknown-field handling, and rescue columns.
   - Assemble decoded Kafka keys, values, timestamps, and headers into Fluss rows.
   - Transcode schema-aware records into the table's native indexed or Arrow log format.
   - Add focused resolver, decoder, projection, and Produce handler coverage.
   
   ### Tests
   
   - Kafka module: 93 tests passed.
   - Reactor build, Checkstyle, Spotless, and RAT validation passed.
   
   ### API and Format
   
   The change defines Kafka mapping through table custom properties and writes existing Fluss indexed or Arrow log formats. It does not introduce a new storage format or change the Fluss public client API.
   
   ### Documentation
   
   Adds design and research notes for Kafka JSON-to-Fluss schema mapping, including complex types and rescue behavior.
   
   ### Stack
   
   - PR 1 — #4186
   - PR 2 — #4187
   - PR 3 — #4188
   - **PR 4/4 — this PR**
   
   Generative AI disclosure: Codex (GPT-5.6) assisted with implementation and commit organization. Human review is required before merge.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [kafka] Add schema-aware Produce transcoding [fluss]

Posted by "gyang94 (via GitHub)" <gi...@apache.org>.
gyang94 closed pull request #4189: [kafka] Add schema-aware Produce transcoding
URL: https://github.com/apache/fluss/pull/4189


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org