[jira] [Created] (FLINK-40318) Add a validation entry point for a VARIANT binary

["Ramin Gharib (Jira)" <ji...@apache.org>]

Ramin Gharib created FLINK-40318:
------------------------------------

             Summary: Add a validation entry point for a VARIANT binary
                 Key: FLINK-40318
                 URL: https://issues.apache.org/jira/browse/FLINK-40318
             Project: Flink
          Issue Type: Improvement
          Components: API / Core
            Reporter: Ramin Gharib
            Assignee: Ramin Gharib


Flink has no way to check that a variant binary is well formed before using it. \{{BinaryVariant}} validates only the metadata version, and everything else relies on \{{checkIndex}} failing lazily during access.
 
Spark's \{{VariantUtil}}, which \{{BinaryVariantUtil}} is derived from, provides \{{isValidVariant(value, metadata)}} backed by a recursive \{{validateImpl}}. It checks the metadata version, structural well formedness of every value, access bounds, primitive type-info validity, dictionary-id existence for object fields, and recurses through nested objects and arrays. Neither method was carried over.
 
This matters once variant bytes arrive from outside the job, for example from a Parquet or Avro source or from restored state, where malformed input should be rejected up front with a clear error instead of surfacing as an index failure somewhere deep in a query.
 
h3. Scope
Port \{{isValidVariant}} and \{{validateImpl}}, and decide where to call them, most likely when constructing a \{{BinaryVariant}} from external bytes rather than on every access.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)