You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "Yufei Zhang (Jira)" <ji...@apache.org> on 2022/01/18 06:39:00 UTC

[jira] [Commented] (FLINK-25686) Support Pulsar Schema evolution in Pulsar Connector

    [ https://issues.apache.org/jira/browse/FLINK-25686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17477605#comment-17477605 ] 

Yufei Zhang commented on FLINK-25686:
-------------------------------------

Please assign this ticket to me [~fpaul] ~ Thank you !

> Support Pulsar Schema evolution in Pulsar Connector
> ---------------------------------------------------
>
>                 Key: FLINK-25686
>                 URL: https://issues.apache.org/jira/browse/FLINK-25686
>             Project: Flink
>          Issue Type: New Feature
>          Components: Connectors / Pulsar
>            Reporter: Yufei Zhang
>            Priority: Minor
>              Labels: Pulsar
>
> Currently the source connector reads all messages as Message<byte[]>, and implements a deserialization abstraction to deserialize the byte array message. This deserialization abstraction supports both flink schema and pulsar schema. However because of this extra layer of abstraction, even when using pulsar schema, the schema info is not provided to the consumer (client), so it is not using any pulsar’s schema validation mechanism. (If pulsar client has a schema, it will send the schema to broker on connect and broker will validate if it is a valid schema according to a preset compatibility rule :[ https://pulsar.apache.org/docs/en/schema-evolution-compatibility/|https://pulsar.apache.org/docs/en/schema-evolution-compatibility/]).  we need to support this schema evolution



--
This message was sent by Atlassian Jira
(v8.20.1#820001)