[jira] [Updated] (FLINK-40240) Support TRY_CAST, IFNULL and NULLIF in Flink CDC YAML Transform

["Mao Jiayi (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-40240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mao Jiayi updated FLINK-40240:
------------------------------
    Description: 
Flink CDC YAML Transform currently supports {{CAST}}, {{COALESCE}}, {{IF}}, and {{CASE}}, but common Flink SQL functions {{TRY_CAST}}, {{IFNULL}}, and {{NULLIF}} cannot be used consistently. {{TRY_CAST}} and {{IFNULL}} are unavailable, while {{NULLIF}} can be parsed but does not have complete validation and runtime support.

This prevents users from safely handling malformed source values with {{TRY_CAST}} and requires verbose alternative expressions for common NULL handling. It also creates an inconsistency between Flink SQL and YAML Transform expressions.

h2. Expected Behavior

The three functions should be available in both projection and filter expressions with Flink SQL-compatible behavior. {{TRY_CAST}} should return {{NULL}} for malformed data on supported conversion paths without changing existing {{CAST}} behavior, while unsupported conversion paths should still be rejected. {{IFNULL}} and {{NULLIF}} should follow Flink SQL value, type inference, and nullability semantics.

  was:
Flink CDC YAML Transform currently supports REGEXP_REPLACE, but the other regular expression functions commonly available in Flink SQL are missing.

This limits the ability to extract, locate, and count pattern matches in row-level transform expressions. Users currently need to implement UDFs for these operations.

The missing functions include:

* REGEXP_EXTRACT
* REGEXP_EXTRACT_ALL
* REGEXP_COUNT
* REGEXP_INSTR
* REGEXP_SUBSTR

YAML Transform should support these functions with behavior aligned with Flink SQL, including Java regular expression syntax, capture group indexes, NULL arguments, invalid patterns, and unmatched input.

REGEXP_EXTRACT_ALL should return ARRAY<STRING> and work correctly in YAML Transform projections.

        Summary: Support TRY_CAST, IFNULL and NULLIF in Flink CDC YAML Transform  (was: Support additional Flink SQL regular expression functions in YAML Transform)

> Support TRY_CAST, IFNULL and NULLIF in Flink CDC YAML Transform
> ---------------------------------------------------------------
>
>                 Key: FLINK-40240
>                 URL: https://issues.apache.org/jira/browse/FLINK-40240
>             Project: Flink
>          Issue Type: Improvement
>          Components: Flink CDC
>            Reporter: Mao Jiayi
>            Priority: Major
>              Labels: pull-request-available
>
> Flink CDC YAML Transform currently supports {{CAST}}, {{COALESCE}}, {{IF}}, and {{CASE}}, but common Flink SQL functions {{TRY_CAST}}, {{IFNULL}}, and {{NULLIF}} cannot be used consistently. {{TRY_CAST}} and {{IFNULL}} are unavailable, while {{NULLIF}} can be parsed but does not have complete validation and runtime support.
> This prevents users from safely handling malformed source values with {{TRY_CAST}} and requires verbose alternative expressions for common NULL handling. It also creates an inconsistency between Flink SQL and YAML Transform expressions.
> h2. Expected Behavior
> The three functions should be available in both projection and filter expressions with Flink SQL-compatible behavior. {{TRY_CAST}} should return {{NULL}} for malformed data on supported conversion paths without changing existing {{CAST}} behavior, while unsupported conversion paths should still be rejected. {{IFNULL}} and {{NULLIF}} should follow Flink SQL value, type inference, and nullability semantics.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)