You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "hudi-bot (via GitHub)" <gi...@apache.org> on 2025/11/30 06:06:39 UTC

[I] Handle JSON schema payload in JsonKafkaSource [hudi]

hudi-bot opened a new issue, #15572:
URL: https://github.com/apache/hudi/issues/15572

   ## JIRA info
   
   - Link: https://issues.apache.org/jira/browse/HUDI-6002
   - Type: Improvement


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Handle JSON schema payload in JsonKafkaSource [hudi]

Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on issue #15572:
URL: https://github.com/apache/hudi/issues/15572#issuecomment-4795447104

   This appears to be addressed on current `master`, though via a different design than the linked PR #8328 (which was closed unmerged).
   
   HUDI-9061 (#12864, commit `b359f76e`) made the Kafka value deserializer in `JsonKafkaSource` pluggable via `hoodie.deltastreamer.source.kafka.json.value.deserializer.class` (default: `StringDeserializer`). With this in place, users can configure a Confluent `KafkaJsonSchemaDeserializer` (or any deserializer that strips the magic-byte / schema-id prefix), and `JsonKafkaSource#getValueAsString` will serialize the resulting object back to a JSON string via Jackson before downstream processing.
   
   Combined with:
   - `JsonSchemaProvider` being registered in `SchemaRegistryProvider` (HUDI-2608, #7727), and
   - `JsonToAvroSchemaConverter` for converting JSON schema → Avro (HUDI-9061),
   
   the JSON-schema payload ingestion path is wireable end-to-end. Could the reporter confirm whether this resolves the original need? If so, this issue can likely be closed.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Handle JSON schema payload in JsonKafkaSource [hudi]

Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan closed issue #15572: Handle JSON schema payload in JsonKafkaSource
URL: https://github.com/apache/hudi/issues/15572


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Handle JSON schema payload in JsonKafkaSource [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on issue #15572:
URL: https://github.com/apache/hudi/issues/15572#issuecomment-3630160022

   ### Linked PR(s)
   
   - https://github.com/apache/hudi/pull/8328


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org