[jira] [Commented] (NIFI-16349) StandardProtobufReader fails to compile Confluent Protobuf schemas with implicit field_meta options

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-16349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18118442#comment-18118442 ] 

ASF subversion and git services commented on NIFI-16349:
--------------------------------------------------------

Commit 4abef54d295f2c3f55c8d09fb1e61f78625259eb in nifi's branch refs/heads/main from Alaksiej Ščarbaty
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=4abef54d295 ]

NIFI-16349 Strip custom Protobuf options before Wire schema compilation (#11710)

Signed-off-by: David Handermann <ex...@apache.org>

> StandardProtobufReader fails to compile Confluent Protobuf schemas with implicit field_meta options
> ---------------------------------------------------------------------------------------------------
>
>                 Key: NIFI-16349
>                 URL: https://issues.apache.org/jira/browse/NIFI-16349
>             Project: Apache NiFi
>          Issue Type: Bug
>            Reporter: Wojciech Trefon
>            Assignee: Alaksiej Ščarbaty
>            Priority: Major
>          Time Spent: 40m
>  Remaining Estimate: 0h
>
> h2. Summary
> {{StandardProtobufReader}} fails to compile Protobuf schemas retrieved from Confluent Schema Registry when a schema contains the {{(confluent.field_meta)}} custom option without an explicit {{confluent/meta.proto}} import.
> The failure occurs during schema compilation, before the message payload is decoded.
> h2. Background
> Confluent Schema Registry treats {{confluent/meta.proto}} as a built-in known dependency. Schemas generated from Kafka Connect logical types can therefore contain {{confluent.field_meta}} without listing {{confluent/meta.proto}} as a normal Schema Registry reference or explicit import.
> h2. Example schema
> {code:protobuf}
> syntax = "proto3";
> package example;
> message Value {
>   optional int32 enabled = 31 [(confluent.field_meta) = {
>     params: [
>       {
>         key: "connect.type",
>         value: "int16"
>       }
>     ]
>   }];
> }
> {code}
> h2. Steps to reproduce
> # Store or generate a Protobuf schema in Confluent Schema Registry that contains {{confluent.field_meta}} without an explicit {{confluent/meta.proto}} import or schema reference.
> # Serialize a message using the Confluent Protobuf wire format and the affected schema.
> # Configure {{StandardProtobufReader}} to use {{ConfluentEncodedSchemaReferenceReader}}, {{ConfluentSchemaRegistry}}, and {{ConfluentProtobufMessageNameResolver}}.
> # Read the serialized message.
> h2. Actual behavior
> Schema compilation fails with an error similar to:
> {code}
> com.squareup.wire.schema.SchemaException: unable to resolve option confluent.field_meta
>   for field enabled
>   in message example.Value
> {code}
> The message payload is not decoded.
> h2. Expected behavior
> {{StandardProtobufReader}} should be able to compile and read a schema accepted by Confluent Schema Registry when the schema uses Confluent's built-in metadata options.
> h2. Impact
> Messages using affected schemas cannot be consumed as records. Existing messages remain affected because their Confluent wire headers continue to reference the original schema identifiers.
> h2. Technical context
> {{ProtobufSchemaCompiler}} provides selected Wire runtime definitions for standard Protobuf types. It does not currently make {{confluent/meta.proto}} or its {{google/protobuf/descriptor.proto}} dependency available during compilation. Wire also requires the source schema to import the file that defines a custom option.
> Observed with the {{StandardProtobufReader}} implementation using Square Wire 6.4.6.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)