[jira] [Updated] (FLINK-36004) Flink SQL returns wrong results for Paimon tables with complex schemas

["Xingcan Cui (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-36004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Xingcan Cui updated FLINK-36004:
--------------------------------
    Attachment: screenshot-1.png

> Flink SQL returns wrong results for Paimon tables with complex schemas
> ----------------------------------------------------------------------
>
>                 Key: FLINK-36004
>                 URL: https://issues.apache.org/jira/browse/FLINK-36004
>             Project: Flink
>          Issue Type: Bug
>          Components: Table SQL / Runtime
>    Affects Versions: 1.18.1, 1.19.1
>            Reporter: Xingcan Cui
>            Priority: Blocker
>         Attachments: screenshot-1.png
>
>
> We have a Paimon table with some nested files such as the following one.
> {code:java}
> `f1` ROW <
>             `f2` ROW <
>                 `f3` ARRAY < ROW < `key` INT, `value` FLOAT > NOT NULL >,
>                 `f4` ARRAY < ROW < `key` INT, `value` STRING > NOT NULL >,
>                 `f5` ARRAY < ROW < `key` BIGINT, `value` FLOAT > NOT NULL >,
>                 `f6` ARRAY < ROW < `key` BIGINT, `value` BIGINT > NOT NULL >,
>                 `f7` ARRAY < ROW < `key` BIGINT, `value` ROW < `f71` ARRAY < FLOAT > > > NOT NULL >,
>                 `f8` ARRAY < ROW < `f81` STRING, `f82` STRING, `f83` BIGINT > NOT NULL >,
>                 `f9` ARRAY < ROW < `key` BIGINT, `value` ROW < `f91` ARRAY < BIGINT > > > NOT NULL >,
>                 `f10` ROW < `f101` ARRAY < ROW < `f102` BIGINT, `f103` STRING, `f104` BIGINT > NOT NULL > >,
>                 `f11` ARRAY < ROW < `key` BIGINT, `value` STRING > NOT NULL >
>             >
> {code}
> When a select query includes some nested columns, the results will be wrong.
> For example, {{SELECT CARDINALITY(f1.f2.f3) AS r FROM...WHERE...}} can return correct results but {{SELECT CARDINALITY(f1.f2.f3) AS r, f1 FROM...WHERE...}} will return wrong values for {{r.}}
> The query execution won't throw any exception but fails silently.
> I'm not sure if this is a Paimon-specific issue, but I also tested running the same query with Spark and StarRocks, and both of them can produce correct results.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)