[jira] [Commented] (IMPALA-15495) Support UPDATE on Iceberg REST catalog tables

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-15495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18129481#comment-18129481 ] 

ASF subversion and git services commented on IMPALA-15495:
----------------------------------------------------------

Commit c8b4c3664ce40f792aa33b4cacb4c4b15e151924 in impala's branch refs/heads/master from Nandor Kollar
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=c8b4c3664c ]

IMPALA-15495: Enable UPDATE on Iceberg REST catalog tables

The REST finalization infrastructure (IMPALA-15310) was built to be
operation-agnostic, but UPDATE was gated by two explicit policy checks.
This patch opens those gates while keeping DELETE, MERGE and OPTIMIZE
blocked.

Analysis-time gate (Analyzer.java): ensureTableWriteSupported() now
accepts a boolean indicating whether the caller is a supported REST
DML operation. ModifyStmt gets it from a new abstract method,
isRestDmlSupported(), which UpdateStmt overrides to return true and
DeleteStmt to return false. MergeStmt and
OptimizeStmt continue to call checkTableCapability(WRITE) which
rejects all non-INSERT DML on REST tables. The method
supportsInsert() is renamed to supportsDml() on FeIcebergTable
since it now gates all supported REST DML, not just INSERT. The helper
ensureRestCatalogInsertEnabled() is renamed to
ensureRestCatalogDmlEnabled().

Commit-time gate (IcebergDmlFinalizer.java): validateRestDmlOperation()
uses a SUPPORTED_REST_OPS EnumSet to accept both INSERT and UPDATE.

Row-lineage virtual columns (IcebergMetaProvider.java): REST tables
never exposed _row_id and _last_updated_sequence_number, because
IcebergMetaProvider kept its own hard-coded list of Iceberg virtual
columns, which missed the format-version 3 additions that IcebergTable
has. Both now use VirtualColumn.getIcebergVirtualColumns(formatVersion),
backed by immutable ICEBERG_VIRTUAL_COLUMNS and
ICEBERG_V3_VIRTUAL_COLUMNS lists, so the two paths cannot drift again.
This also fixes SELECT of these columns on V3 REST tables.

No backend changes are needed: the C++ sinks are catalog-agnostic,
the coordinator-side finalization routing is based on rest_catalog_name
(not operation type), and IcebergCatalogOpExecutor.updateRows() already
uses the pure Iceberg API (txn.newRowDelta()).

Testing:
- AnalyzerCapabilityTest updated for renamed methods and new messages.
- IcebergRestDmlOperationTest updated: UPDATE validation passes and
  DELETE is rejected.
- test_rest_catalog_insert_into: UPDATE removed from the unsupported
  list.
- New test_rest_catalog_update: runs iceberg-rest-update.test, then
  injects CommitStateUnknownException (files are kept) and
  CommitFailedException (written files are cleaned up) for UPDATE.
- New iceberg-rest-update.test adapted from iceberg-v3-update.test:
  basic UPDATE, multi-row, _row_id and _last_updated_sequence_number
  verification, DV file checks, single-column, all-types (individual
  and multi-column), JOIN-based UPDATE FROM, duplicate error, NULL
  handling, column swap, cross-column expression, and
  SKIP_UNNEEDED_UPDATES_COL_LIMIT.
- New iceberg-rest-update-partitions.test adapted from
  iceberg-v3-update-partitions.test: identity, truncate,
  bucket+truncate, date year/month/day, timestamp year/month/day/hour,
  and numeric truncate partition transforms. Partition evolution tests
  are omitted as ALTER TABLE DDL is not available on REST-backed tables.
- Tables are created via the REST API since DDL is not supported.

Change-Id: I847f051e9a350b730935326c983c7459215c54e2
Assisted-by: Claude Opus 4.6 (Claude Code)
Reviewed-on: http://gerrit.cloudera.org:8080/25046
Tested-by: Impala Public Jenkins <im...@cloudera.com>
Reviewed-by: Peter Rozsa <pr...@cloudera.com>


> Support UPDATE on Iceberg REST catalog tables
> ---------------------------------------------
>
>                 Key: IMPALA-15495
>                 URL: https://issues.apache.org/jira/browse/IMPALA-15495
>             Project: IMPALA
>          Issue Type: Sub-task
>            Reporter: Peter Rozsa
>            Assignee: Nándor Kollár
>            Priority: Major
>              Labels: impala-iceberg
>
> UPDATE is rejected in ModifyStmt.
> The commit logic is already pure Iceberg: IcebergCatalogOpExecutor.updateRows builds a RowDelta.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org