You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Bettle (Jira)" <ji...@apache.org> on 2026/09/14 16:45:00 UTC

[jira] [Created] (FLINK-40658) UNNEST silently drops null ROW elements from ARRAY

Bettle created FLINK-40658:
------------------------------

             Summary: UNNEST silently drops null ROW elements from ARRAY
                 Key: FLINK-40658
                 URL: https://issues.apache.org/jira/browse/FLINK-40658
             Project: Flink
          Issue Type: Bug
          Components: Table SQL / Runtime
    Affects Versions: 2.4.0
            Reporter: Bettle


While implementing DataFrame.explode() for FLINK-40435, I noticed that UNNEST silently drops null ROW elements from an ARRAY.

This reproduces through direct Table API/SQL calls on an unchanged baseline, without importing or using the new DataFrame implementation.

h3. Reproduction

Run the following code in a configured PyFlink environment:

{code:python}
from pyflink.table import EnvironmentSettings, TableEnvironment

t_env = TableEnvironment.create(EnvironmentSettings.in_batch_mode())
source = t_env.sql_query(
    "SELECT ARRAY[CAST(NULL AS ROW<n INT, s STRING>), "
    "ROW(1, 'a')] AS items"
)

for join, condition in [
    ("CROSS JOIN", ""),
    ("LEFT JOIN", " ON TRUE"),
]:
    query = (
        f"SELECT expanded.* FROM `{str(source)}` AS src "
        f"{join} UNNEST(src.items) AS expanded(n, s){condition}"
    )
    with t_env.sql_query(query).execute().collect() as rows:
        actual = list(rows)
    print(join, actual)
{code}

h3. Expected result

Both join variants should return two rows, regardless of order:

{code}
(NULL, NULL)
(1, 'a')
{code}

h3. Actual result

Both join variants return only:

{code}
(1, 'a')
{code}

Execution completes without an exception, but the null ROW element is missing.

The input is a nonempty array containing a null ROW element. This is different from an empty array, a null array, or a non-null ROW whose fields are all null.

h3. Environment

* Flink 2.4-SNAPSHOT, baseline e01bbcacd96
* Batch mode
* Java 17.0.14
* Python 3.12.11

The equivalent direct Table API/SQL tests reproduced this behavior for both CROSS JOIN and LEFT JOIN. Released versions and streaming mode have not been verified for this issue.

h3. Related work

This was discovered during FLINK-40435, but reproduces without the new DataFrame wrappers. I am reporting it separately to keep that work focused on the DataFrame API. No underlying runtime fix is included in that work.

If this is confirmed to be a bug that needs fixing, I would be happy to investigate further and try to contribute a fix.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)