You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2023/06/16 22:41:00 UTC
[jira] [Commented] (IMPALA-12197) Time Travel Queries on Iceberg tables fail with "ERROR: IllegalArgumentException: null"
[ https://issues.apache.org/jira/browse/IMPALA-12197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17733673#comment-17733673 ]
ASF subversion and git services commented on IMPALA-12197:
----------------------------------------------------------
Commit 06eb62d3efa1c94810c4276f90896fa62205a49b in impala's branch refs/heads/master from Andrew Sherman
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=06eb62d3e ]
IMPALA-12197: Prevent assertion failures when isClusteringColumn() is called on a IcebergTimeTravelTable.
When using local catalog mode, if a runtime filter is being generated
for a time travel iceberg table, then a query may fail with "ERROR:
IllegalArgumentException: null"
In the planner an Iceberg table that is being accessed with Time Travel
is represented by an IcebergTimeTravelTable object. This object
represents a time-based variation on a base table. The
IcebergTimeTravelTable may represent a different schema from the base
table, it does this by tracking its own set of Columns. As part of
generating a runtime filter the isClusteringColumn() method is called
on the table. IcebergTimeTravelTable was delegating this call to the
base object. In local catalog mode this method is implemented by
LocalTable which has a Preconditions check (an assertion) that the
column parameter matches the stored column. In this case the check
fails as the base table and time travel table have their own distinct
set of column objects.
The fix is to have IcebergTimeTravelTable provide its own
isClusteringColumn() method. For iceberg there are no clustering
columns, so this method simply returns false.
TESTING
- Ran all end-to-end tests.
- Added test case for query that failed.
Change-Id: I51d04c8757fb48bd417248492d4615ac58085632
Reviewed-on: http://gerrit.cloudera.org:8080/20034
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Time Travel Queries on Iceberg tables fail with "ERROR: IllegalArgumentException: null"
> ---------------------------------------------------------------------------------------
>
> Key: IMPALA-12197
> URL: https://issues.apache.org/jira/browse/IMPALA-12197
> Project: IMPALA
> Issue Type: Bug
> Affects Versions: Impala 2.13.0
> Reporter: Andrew Sherman
> Assignee: Andrew Sherman
> Priority: Critical
>
> When using local catalog mode, if a runtime filter is being generated for a time travel iceberg table, then a query may fail with "ERROR: IllegalArgumentException: null"
> In the planner an Iceberg table that is being accessed with Time Travel is represented by an IcebergTimeTravelTable object. This object represents a time-based variation on a base table. The IcebergTimeTravelTable may represent a different schema from the base table, it does this by tracking its own set of Columns. As part of generating a runtime filter the isClusteringColumn() method is called on the table. IcebergTimeTravelTable was delegating this call to the base object. In local catalog mode this method is implemented by LocalTable which has a Preconditions check (an assertion) that the column parameter matches the stored column. In this case the check fails as the base table and time travel table have their own distinct set of column objects.
> A possible fix is to have IcebergTimeTravelTable provide its own isClusteringColumn() method. For iceberg there are no clustering columns so this method simply returns false.
> {code}
> I0608 18:19:04.009989 26712 jni-util.cc:288] 30478706c6b2ab5b:ac06893500000000] java.lang.IllegalArgumentException
> at com.google.common.base.Preconditions.checkArgument(Preconditions.java:131)
> at org.apache.impala.catalog.local.LocalTable$ColumnMap.isClusteringColumn(LocalTable.java:427)
> at org.apache.impala.catalog.local.LocalTable.isClusteringColumn(LocalTable.java:286)
> at org.apache.impala.catalog.local.LocalIcebergTable.isClusteringColumn(LocalIcebergTable.java:59)
> at org.apache.impala.catalog.ForwardingFeIcebergTable.isClusteringColumn(IcebergTimeTravelTable.java:546)
> at org.apache.impala.catalog.IcebergTimeTravelTable.isClusteringColumn(IcebergTimeTravelTable.java:70)
> at org.apache.impala.planner.RuntimeFilterGenerator.isBoundByPartitionColumns(RuntimeFilterGenerator.java:1088)
> at org.apache.impala.planner.RuntimeFilterGenerator.assignRuntimeFilters(RuntimeFilterGenerator.java:964)
> at org.apache.impala.planner.RuntimeFilterGenerator.generateFilters(RuntimeFilterGenerator.java:863)
> at org.apache.impala.planner.RuntimeFilterGenerator.generateFilters(RuntimeFilterGenerator.java:866)
> at org.apache.impala.planner.RuntimeFilterGenerator.generateFilters(RuntimeFilterGenerator.java:866)
> at org.apache.impala.planner.RuntimeFilterGenerator.generateFilters(RuntimeFilterGenerator.java:866)
> at org.apache.impala.planner.RuntimeFilterGenerator.generateFilters(RuntimeFilterGenerator.java:856)
> at org.apache.impala.planner.RuntimeFilterGenerator.generateFilters(RuntimeFilterGenerator.java:866)
> at org.apache.impala.planner.RuntimeFilterGenerator.generateRuntimeFilters(RuntimeFilterGenerator.java:735)
> at org.apache.impala.planner.Planner.createPlanFragments(Planner.java:147)
> at org.apache.impala.planner.Planner.createPlans(Planner.java:251)
> at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1772)
> at org.apache.impala.service.Frontend.getPlannedExecRequest(Frontend.java:2603)
> at org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:2432)
> at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:2056)
> at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1831)
> at org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:164)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org