You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "SteveStevenpoor (via GitHub)" <gi...@apache.org> on 2026/06/01 10:41:27 UTC
[I] Flink: Support ALTER TABLE ... DROP PARTITION syntax [iceberg]
SteveStevenpoor opened a new issue, #16646:
URL: https://github.com/apache/iceberg/issues/16646
### Feature Request / Improvement
## Motivation
Flink already supports ```ALTER TABLE ... DROP PARTITION``` and invokes the catalog's ```dropPartition(...)``` implementation. However, the Iceberg Flink catalog does not currently implement this operation, leaving users without a SQL-level mechanism to remove all data in a partition.
### How users drop a partition today
DELETE is not supported by the Iceberg Flink catalog for batch or streaming. Users must drop into the Java/Scala Table API (table.newDelete().deleteFromRowFilter(...).commit()) or move the operation outside SQL entirely.
## Proposed feature
Add support for the ```ALTER TABLE … DROP PARTITION``` syntax, addressing the partition field name, in the same human-readable form Iceberg already writes to manifest partition paths:
```sql
ALTER TABLE events DROP PARTITION (id = 0);
ALTER TABLE events DROP PARTITION (region = 'eu', dt = '2024-01-01');
ALTER TABLE events DROP IF EXISTS PARTITION (data = 'z')
```
## Implementation sketch
- Flink: ```FlinkCatalog.dropPartition(...)``` translates the partition specification into an Iceberg row filter and invokes ```deleteFromRowFilter(...)```
- Flink's SQL parser already accepts ```ALTER TABLE … DROP [IF EXISTS] PARTITION (...)```.
## Safety
```deleteFromRowFilter``` already runs the strict-projection check at commit, so partial-file deletes are rejected by Iceberg core; the new API surface cannot silently remove rows outside the targeted partition.
### Query engine
Flink
### Willingness to contribute
- [x] I can contribute this improvement/feature independently
- [ ] I would be willing to contribute this improvement/feature with guidance from the Iceberg community
- [ ] I cannot contribute this improvement/feature at this time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Flink: Support ALTER TABLE ... DROP PARTITION syntax [iceberg]
Posted by "kaiwangleo (via GitHub)" <gi...@apache.org>.
kaiwangleo commented on issue #16646:
URL: https://github.com/apache/iceberg/issues/16646#issuecomment-5611338923
Thanks for opening this issue. I implemented `ALTER TABLE ... DROP PARTITION` support in the Iceberg Flink catalog.
The implementation:
- translates `CatalogPartitionSpec` using partition field names into an Iceberg row filter;
- supports identity partition fields and rejects unsupported transforms with a clear catalog error;
- handles typed partition values, including integer, date, timestamp/time, binary/fixed, and null values;
- checks partition existence before deleting, so `DROP IF EXISTS` does not create an unnecessary snapshot;
- aligns `listPartitions` values with the format accepted by `DROP PARTITION`;
- is implemented consistently for Flink 1.20, 2.1, 2.2, and 2.3.
Tests cover SQL execution, multiple fields, typed and null values, missing partitions/tables, unpartitioned tables, unsupported transforms, and the no-op `DROP IF EXISTS` path.
Validation completed:
- all four Flink version test tasks passed;
- Gradle reported `BUILD SUCCESSFUL` with 68 actionable tasks;
- `git diff --check` passed.
The local Spotless check could not be started because the Windows checkout's CRLF-formatted Gradle wrapper cannot be executed from the WSL mount; this is an environment limitation rather than a test or compilation failure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Flink: Support ALTER TABLE ... DROP PARTITION syntax [iceberg]
Posted by "kaiwangleo (via GitHub)" <gi...@apache.org>.
kaiwangleo commented on issue #16646:
URL: https://github.com/apache/iceberg/issues/16646#issuecomment-5628425799
Thanks for the discussion and for raising the concerns about exposing `DROP PARTITION` through the default Flink catalog.
I reviewed the related dev@iceberg discussion. The main concerns are that Iceberg uses hidden partitioning, `DROP PARTITION` has no corresponding `ADD PARTITION` operation, and the feature may be more appropriate for Hive-to-Iceberg migration tooling or a connector-specific extension. The discussion also noted that partition evolution requires additional consideration.
The current implementation is intentionally limited to identity partition fields. It translates the Flink partition specification into an Iceberg row filter, relies on `deleteFromRowFilter` for strict projection validation, and avoids creating a snapshot when `DROP IF EXISTS` targets a missing partition. Tests cover typed values, null values, missing tables or partitions, unsupported transforms, and all supported Flink versions.
However, given the lack of clear community consensus and the concern about adding this behavior to the default `FlinkCatalog`, I think the API placement and semantics should be confirmed before merging. In particular, we should decide whether this belongs in the default catalog, a migration-specific tool, or a connector-specific extension, and document the expected behavior across partition evolution.
Would maintainers prefer to continue with the default Flink catalog implementation, or should this work be redirected to a use-case-specific extension?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Flink: Support ALTER TABLE ... DROP PARTITION syntax [iceberg]
Posted by "kaiwangleo (via GitHub)" <gi...@apache.org>.
kaiwangleo commented on issue #16646:
URL: https://github.com/apache/iceberg/issues/16646#issuecomment-5611385246
The previously related PR #16647 was closed on July 12, 2026 due to inactivity. A new PR, #18040, has been submitted to implement this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Flink: Support ALTER TABLE ... DROP PARTITION syntax [iceberg]
Posted by "pvary (via GitHub)" <gi...@apache.org>.
pvary commented on issue #16646:
URL: https://github.com/apache/iceberg/issues/16646#issuecomment-5616898751
In the Iceberg community there are mixed feelings around dropping partitions. See: https://lists.apache.org/thread/7tdkrkxy40y26f9d88mqlxgr0qdowwjq
I'm not sure we would like to provide this functionality through FlinkCatalog either.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org