[jira] [Closed] (FLINK-28609) Flink-Pulsar connector fails on larger schemas

["Zili Chen (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-28609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zili Chen closed FLINK-28609.
-----------------------------
    Resolution: Fixed

master via d9bcbffc006481c09a8d2e04aa05cc92cd5c80d2
1.15 via cbd6c8d47b43f67c15d3a7d7bfb9b42b3e7d033b
1.14 via 55cf3b37bc6c864cc6ce5b7e9913de2de00b95cb

> Flink-Pulsar connector fails on larger schemas
> ----------------------------------------------
>
>                 Key: FLINK-28609
>                 URL: https://issues.apache.org/jira/browse/FLINK-28609
>             Project: Flink
>          Issue Type: Bug
>          Components: Connectors / Pulsar
>    Affects Versions: 1.14.3, 1.14.4, 1.14.5, 1.15.1
>            Reporter: Jacek Wislicki
>            Assignee: Yufan Sheng
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 1.16.0, 1.14.6, 1.15.3
>
>         Attachments: [FLINK-28609][Connector_Pulsar]_PulsarSchema_didn't_get_properly_serialized_.patch, exception.txt
>
>
> When a model results in a larger schema (this seems to be related to its byte array representation), the number of expected bytes to read is different than the number of actually read bytes: [^exception.txt]. The "read" is such a case is always 1018 while the expected "byteLen" gives a greater value. For smaller schemata, the numbers are equal (less than 1018) and no issue occurs.
> The problem reproduction is on [GitHub|https://github.com/JacekWislicki/vp-test2]. There are 2 simple jobs (SimpleJob1 and SimpleJob2) using basic models for the Pulsar source definition (PulsarMessage1 and PulsarMessage2, respectively). Each of the corresponding schemata is properly serialised and deserialised, unless an effective byte array length becomes excessive (marked with "the problem begins" in model classes). The fail condition can be achieved by a number of fields (PulsarMessage1) or just longer field names (PulsarMessage2). The problem occurs on either Avro or a JSON schema set in the Pulsar source definition.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)