You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "nazq (via GitHub)" <gi...@apache.org> on 2026/06/01 19:50:06 UTC
[PR] feat(schema): add rename_column to UpdateSchemaAction [iceberg-rust]
nazq opened a new pull request, #2563:
URL: https://github.com/apache/iceberg-rust/pull/2563
## Which issue does this PR close?
- Closes #2562.
## What changes are included in this PR?
Adds `rename_column` to `UpdateSchemaAction`, completing the add/delete/rename triad for non-incompatible schema evolution in `transaction::update_schema`.
### Public API
```rust
let tx = Transaction::new(&table);
let action = tx.update_schema()
.add_column(AddColumn::optional("new_col", Type::Primitive(PrimitiveType::Int)))
.rename_column("old_name", "new_name")
.rename_column("person.name", "fullname") // nested supported
.delete_column("dead_col");
let tx = action.apply(tx).unwrap();
let table = tx.commit(&catalog).await.unwrap();
```
`rename_column(path_from, new_name)` mirrors the existing `add_column` / `delete_column` builder shape. Field IDs are preserved across rename — only the leaf name changes.
### Implementation
A new `renames: Vec<(String, String)>` accumulator on `UpdateSchemaAction`, plus a new step in `commit()` between deletes and additions:
- **After deletes** so a rename can re-use a name being deleted in the same action.
- **Before additions** so an addition can re-use a name being renamed away.
The renames are resolved to a `HashMap<i32, String>` keyed by field ID, then threaded through `rebuild_fields` / `rebuild_field` — same recursive walk that handles deletes and nested additions. Each rebuild call site that previously copied `field.name.clone()` now looks up the rename map first and falls back to the original name. Identifier-field handling falls out for free because Rust keys `identifier_field_ids` by ID, not name — the existing `with_identifier_field_ids(base_schema.identifier_field_ids())` step propagates the set unchanged.
### Semantics
Modeled on `pyiceberg.table.update.schema.UpdateSchema.rename_column`:
- `path_from` must exist in the current schema → otherwise `PreconditionFailed`.
- A field cannot be both renamed and deleted in the same action → `PreconditionFailed` (intent is ambiguous; matches PyIceberg).
- `new_name` cannot contain `SCHEMA_NAME_DELIMITER` → `PreconditionFailed` (the unqualified-name requirement; preventing it from silently looking like a move-across-structs).
- `new_name` cannot collide with a sibling that is not itself being deleted or renamed away → `PreconditionFailed`.
- Same field renamed twice → last rename wins (matches PyIceberg's "stack on prior update" behavior).
### Scope
`rename_column` only. PyIceberg's `UpdateSchema` has other ops (`update_column`, `make_column_required`, `move_*`, `set_identifier_fields`) — those are out of scope here and would be follow-up PRs if there's interest.
## Are these changes tested?
Yes — 10 new unit tests in `transaction::update_schema::tests`, matching the style and depth of the existing `test_*_add_column*` / `test_*_delete_column*` tests:
| Test | What it covers |
|---|---|
| `test_rename_column` | Simple root-level rename, verifies field ID preserved |
| `test_rename_nested_column` | Nested rename (`person.name` → `person.fullname`), preserves sibling fields |
| `test_rename_missing_column_fails` | Missing source field → `PreconditionFailed` |
| `test_rename_and_delete_same_column_fails` | Rename + delete on same column → `PreconditionFailed` |
| `test_rename_to_existing_sibling_fails` | Collision with a non-deleted, non-renamed sibling → `PreconditionFailed` |
| `test_rename_path_with_dot_in_new_name_fails` | Rejects qualified new name |
| `test_rename_identifier_field_preserves_identifier_ids` | Verifies `identifier_field_ids` is preserved across rename of an identifier field |
| `test_rename_frees_name_for_addition` | Composition test: rename + add re-uses the old name |
| `test_rename_same_column_twice_last_wins` | Idempotency / "last wins" semantics |
| `test_rename_self_is_noop` | Rename to current name produces unchanged schema |
All 18 existing `transaction::update_schema` tests still pass (no regressions). Full `iceberg` lib suite: 1306/1306 passing. Clippy + rustfmt clean.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(schema): add rename_column to UpdateSchemaAction [iceberg-rust]
Posted by "TwinklerG (via GitHub)" <gi...@apache.org>.
TwinklerG commented on PR #2563:
URL: https://github.com/apache/iceberg-rust/pull/2563#issuecomment-4599547031
Implemented most of the schema evolution features in #2451.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(schema): add rename_column to UpdateSchemaAction [iceberg-rust]
Posted by "nazq (via GitHub)" <gi...@apache.org>.
nazq commented on PR #2563:
URL: https://github.com/apache/iceberg-rust/pull/2563#issuecomment-4649681507
Thanks for catching that and even opening a fix, @TwinklerG — much appreciated. I rebased the branch onto current \`main\` and regenerated \`public-api.txt\` with \`make generate-public-api\`; it adds exactly the three \`RenameColumn\` lines you had. \`make check-public-api\` is green locally now. Closing your courtesy PR since this folds the same change into the feature commit — thank you again. 🙏
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(schema): add rename_column to UpdateSchemaAction [iceberg-rust]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #2563:
URL: https://github.com/apache/iceberg-rust/pull/2563#issuecomment-4997799668
This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(schema): add rename_column to UpdateSchemaAction [iceberg-rust]
Posted by "nazq (via GitHub)" <gi...@apache.org>.
nazq commented on PR #2563:
URL: https://github.com/apache/iceberg-rust/pull/2563#issuecomment-4604014771
Thanks for pointing at #2451, @TwinklerG — I went and read through it carefully.
After studying both PRs, I think your refactor is the more architecturally sound long-term direction. The operations-buffer-then-`schema_update` shape is what lets `UpdateSchemaAction` correctly model dependent operations (rename-then-update, add-then-make-required, etc.) — and it's the shape that mirrors what @Fokko built in PyIceberg's `UpdateSchema` and the Java `SchemaUpdate`, which is exactly what #697 asks for. The current `add_column` / `delete_column` style from #2120 can't grow into that without a breaking change, so paying the API churn cost once now is the right call.
To make that easier, I've **updated this PR to use the `RenameColumn` typed-builder shape from your #2451**, so the user-facing call is now identical between the two:
```rust
action.rename(RenameColumn::builder().name("old").new_name("new").build())
```
That way this PR functions as a clean stepping stone: callers adopting it today won't need a source change when #2451 lands — only the dispatch underneath `.rename()` changes (imperative-vec → operations-buffer). If maintainers want a lighter incremental win now, this PR is ready; either way, **#2451 is the destination**.
We've been carrying this on a fork for a downstream consumer; happy to migrate to #2451's full API as soon as it merges. If contributing test coverage for any of your `#[ignore = "not yet implemented"]` cases would help land it faster, let me know which ones — we have real-world data for case-insensitive resolution and decimal-precision widening that could plug in.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(schema): add rename_column to UpdateSchemaAction [iceberg-rust]
Posted by "TwinklerG (via GitHub)" <gi...@apache.org>.
TwinklerG commented on PR #2563:
URL: https://github.com/apache/iceberg-rust/pull/2563#issuecomment-4645074671
Hey @nazq , maybe you need to run `make generate-public-api` to make the pipline happy.
I have opened a [PR](https://github.com/nazq/iceberg-rust/pull/2) to fix it for your PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(schema): add rename_column to UpdateSchemaAction [iceberg-rust]
Posted by "TwinklerG (via GitHub)" <gi...@apache.org>.
TwinklerG commented on PR #2563:
URL: https://github.com/apache/iceberg-rust/pull/2563#issuecomment-4608502755
Thanks @nazq for taking time to review my PR. Your PR is a brilliant move to ensure a smooth transition. I really appreciate your validation of the architecture in #2451.
Actually the test cases including `#[ignore = "not yet implemented"]` cases are ported from [iceberg-java](https://github.com/apache/iceberg) `core/src/test/java/org/apache/iceberg/TestSchemaUpdate.java`, and I haven't implemented them all yet. I am glad if you can contribute more cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(schema): add rename_column to UpdateSchemaAction [iceberg-rust]
Posted by "TwinklerG (via GitHub)" <gi...@apache.org>.
TwinklerG commented on PR #2563:
URL: https://github.com/apache/iceberg-rust/pull/2563#issuecomment-4613782449
> Hey @TwinklerG, PR'ed the first batch as [TwinklerG#1](https://github.com/TwinklerG/iceberg-rust/pull/1) against #2451's branch. Three of the #[ignore = "not yet implemented"] stubs now have real test bodies and pass against testsHey , 已将第一批 PR 提交至 [TwinklerG#1](https://github.com/TwinklerG/iceberg-rust/pull/1) 的分支，针对 #2451 的分支。三个 #[ignore = "尚未实现"] 的占位符现在有了实际的测试代码，并且通过了测试
>
> * add_required_column_without_default
> * add_required_column_with_default添加带默认值的必需列
> * add_required_column_with_update_column_default添加带更新列默认值的必需列
>
> ~6 more of the ignored tests look doable against the current implementation without needing new features (add_nested_list_of_structs, the *_preserves_other_metadata family, delete_fields). The remaining ~6 are the case-insensitive variants, which depend on a flag schema_update doesn't yet thread through name resolution — that's a real implementation contribution, not just a test.~6 更多被忽略的测试看起来可以在当前实现中完成，而无需新功能（add_nested_list_of_structs、*_preserves_other_metadata 系列、delete_fields）。剩下的 ~6 是不区分大小写的变体，这取决于一个标志，而 schema_update 目前还没有通过名称解析传递该标志——这是一个真正的实现贡献，而不仅仅是测试。
>
> I have some free time later this afternoon so drop a comment if this direction looks sound.我今天下午有些空闲时间，如果这个方向看起来可行，请留言。
>
> Cheers 干杯
Thanks @nazq for the contribution. I will review it soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(schema): add rename_column to UpdateSchemaAction [iceberg-rust]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #2563: feat(schema): add rename_column to UpdateSchemaAction
URL: https://github.com/apache/iceberg-rust/pull/2563
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(schema): add rename_column to UpdateSchemaAction [iceberg-rust]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #2563:
URL: https://github.com/apache/iceberg-rust/pull/2563#issuecomment-4920424034
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(schema): add rename_column to UpdateSchemaAction [iceberg-rust]
Posted by "nazq (via GitHub)" <gi...@apache.org>.
nazq commented on PR #2563:
URL: https://github.com/apache/iceberg-rust/pull/2563#issuecomment-4612613423
Hey @TwinklerG, PR'ed the first batch as TwinklerG/iceberg-rust#1 against #2451's branch. Three of the #[ignore = "not yet implemented"] stubs now have real test bodies and pass against tests
- add_required_column_without_default
- add_required_column_with_default
- add_required_column_with_update_column_default
~6 more of the ignored tests look doable against the current implementation without needing new features (add_nested_list_of_structs, the *_preserves_other_metadata family, delete_fields). The remaining ~6 are the case-insensitive variants, which depend on a flag schema_update doesn't yet thread through name resolution — that's a real implementation contribution, not just a test.
I have some free time later this afternoon so drop a comment if this direction looks sound.
Cheers
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org