[jira] [Updated] (NIFI-14424) Support Confluent schema registry Protobuf wire format in StandardProtobufReader

["Alaksiej Ščarbaty (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/NIFI-14424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Alaksiej Ščarbaty updated NIFI-14424:
-------------------------------------
    Summary: Support Confluent schema registry Protobuf wire format in StandardProtobufReader  (was: Support for Confluent schema registry Protobuf wire format in StandardProtobufReader)

> Support Confluent schema registry Protobuf wire format in StandardProtobufReader
> --------------------------------------------------------------------------------
>
>                 Key: NIFI-14424
>                 URL: https://issues.apache.org/jira/browse/NIFI-14424
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Extensions
>            Reporter: Alaksiej Ščarbaty
>            Assignee: Lukas Kucharski
>            Priority: Major
>             Fix For: 2.6.0
>
>         Attachments: image-2025-07-16-12-52-20-296.png, image-2025-07-17-10-32-37-291.png, image-2025-07-17-14-40-58-260.png, image-2025-07-17-14-42-31-326.png, image-2025-07-17-14-43-58-023.png, image-2025-07-17-14-45-47-577.png, image-2025-07-17-14-51-24-943.png, image-2025-07-17-14-52-18-792.png
>
>          Time Spent: 13h 20m
>  Remaining Estimate: 0h
>
> h2.   Summary
>   Add Confluent Protobuf Wire Format Support - Foundation Services and APIs
> h2.   Description
>   This enhancement introduces foundational support for deserializing Protobuf messages encoded using the Confluent protobuf wire format in NiFi. The existing Confluent Schema Registry integration only supported Avro data decoding. Confluent's Protobuf serialization adds a variable-length byte section to the message header containing message indexes that must be parsed and interpreted.
>   This ticket represents Phase 1 of the implementation, providing the core services and API extensions. Phase 2 will introduce the new StandardProtobufReader component.
> h2.   Key Components Implemented
> h3.   API Extensions (nifi-schema-registry-service-api)
>   - {*}SchemaDefinition{*}: New abstraction representing raw schema forms with schema text and referenced schema collections
>   - {*}MessageNameResolver{*}: Controller service interface for translating encoded Protobuf message indexes into message names
>   - {*}MessageName{*}: Abstraction for values returned by MessageNameResolver
> h3.   Confluent Platform Bundle Enhancements
>   - {*}ConfluentProtobufMessageNameResolver{*}: Service that decodes Confluent-standardized message name indexes from wire format using zigzag encoded varints
>   - Enhanced {*}RestSchemaRegistryClient{*}: Extended to support recursive fetching of referenced/imported Protobuf schemas
>   - {*}ProtobufMessageSchema{*}: Schema representation supporting nested message parsing
>   - ANTLR Parser Integration: Added Proto3 grammar parser for schema structure analysis without requiring full compilation
> h3.   Wire Format Processing
>   - Message Index Decoding: Parses variable-length message indexes (e.g., [0][2][0] or [1]) from Confluent wire format headers
>   - Nested Message Support: Resolves complex message paths like MessageA.MessageE.MessageG
>   - Schema Structure Analysis: Parses Protobuf schemas to locate correct message definitions for deserialization
> h2.   Next Phase
>   Phase 2 (linked issue) will implement *StandardProtobufReader* component that utilizes these foundational services to provide full Protobuf record processing capabilities with Schema Registry integration.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)