You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2025/07/02 11:28:00 UTC
[jira] [Commented] (IMPALA-14185) Error unnesting nested array from Iceberg with DELETE files
[ https://issues.apache.org/jira/browse/IMPALA-14185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17987871#comment-17987871 ]
ASF subversion and git services commented on IMPALA-14185:
----------------------------------------------------------
Commit 191aec5298e6f5e2ca352f19b9abb3b111232554 in impala's branch refs/heads/master from Daniel Becker
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=191aec529 ]
IMPALA-14185: Error unnesting nested array from Iceberg with DELETE files
When trying to doubly unnest a 2D array from an Iceberg table that has
delete files but not for every data file, we run into an error:
Filtering an unnested collection that comes from a UNION [ALL] is not
supported yet.
This is because there is a UNION node because of the Iceberg delete
files, and there is an added "not-empty" conjunct on the collections.
IMPALA-12753 describes a bug where a conjunct on an unnested collection
coming from a UNION ALL is only applied to the first UNION operand. To
avoid incorrectness, we disabled this case in the commit for
IMPALA-12695, but its unintended consequence is that it leads to this
error with Iceberg tables.
However, in this case with Iceberg deletes, the bug described in
IMPALA-12753 is not present because both sides of the UNION have the
same tuple id, so conjuncts are naturally applied to both sides.
This commit relaxes the check, which now does not fire if all UNION
operands have the same tuple ids.
Testing:
- existing tests related to IMPALA-12753 pass
- added a regression test with an Iceberg table with DELETE files
Change-Id: Ifbc6f580586d4b337f33a2f32052aa07f6fca828
Reviewed-on: http://gerrit.cloudera.org:8080/23107
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Error unnesting nested array from Iceberg with DELETE files
> -----------------------------------------------------------
>
> Key: IMPALA-14185
> URL: https://issues.apache.org/jira/browse/IMPALA-14185
> Project: IMPALA
> Issue Type: Bug
> Reporter: Daniel Becker
> Assignee: Daniel Becker
> Priority: Major
> Labels: impala-iceberg, impala-iceberg-active-backlog
>
> When trying to doubly unnest a 2D array from an Iceberg table that has delete files but not for every data file, we run into an error:
> {code:java}
> Filtering an unnested collection that comes from a UNION [ALL] is not supported yet.
> {code}
> This is because there is a UNION node because of the Iceberg files, and there is an added non-empty conjunct on the collections. Therefore we run into the triggering situation of IMPALA-12753. To avoid incorrectness, we disabled this case in the commit for IMPALA-12695, but its unintended consequence is that it leads to this error with Iceberg tables.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org