You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "rmoff (via GitHub)" <gi...@apache.org> on 2025/07/21 15:04:50 UTC
[PR] Flink can now add/drop/modify columns [iceberg]
rmoff opened a new pull request, #13617:
URL: https://github.com/apache/iceberg/pull/13617
Per https://issues.apache.org/jira/browse/FLINK-21634, these features are now available.
e.g.
```sql
Flink SQL> SET 'execution.checkpointing.interval' = '30s';
[INFO] Execute statement succeeded.
Flink SQL> CREATE TABLE test01 (col1 INT)
> WITH (
> 'connector' = 'iceberg',
> 'catalog-name' = 'foo',
> 'catalog-database' = 'my_glue_db',
> 'warehouse' = 's3://rmoff-lakehouse/00/',
> 'catalog-impl' = 'org.apache.iceberg.aws.glue.GlueCatalog',
> 'ioImpl' = 'org.apache.iceberg.aws.s3.S3FileIO');
[INFO] Execute statement succeeded.
-- Add a column
Flink SQL> ALTER TABLE test01 ADD test_col INT;
[INFO] Execute statement succeeded.
Flink SQL> DESCRIBE test01;
+----------+------+------+-----+--------+-----------+
| name | type | null | key | extras | watermark |
+----------+------+------+-----+--------+-----------+
| col1 | INT | TRUE | | | |
| test_col | INT | TRUE | | | |
+----------+------+------+-----+--------+-----------+
2 rows in set
-- Drop a column
Flink SQL> ALTER TABLE test01 DROP test_col;
[INFO] Execute statement succeeded.
Flink SQL> DESCRIBE test01;
+------+------+------+-----+--------+-----------+
| name | type | null | key | extras | watermark |
+------+------+------+-----+--------+-----------+
| col1 | INT | TRUE | | | |
+------+------+------+-----+--------+-----------+
1 row in set
-- Rename a column
Flink SQL> ALTER TABLE test01 RENAME test_col TO foo;
[INFO] Execute statement succeeded.
Flink SQL> DESCRIBE test01;
+------+------+------+-----+--------+-----------+
| name | type | null | key | extras | watermark |
+------+------+------+-----+--------+-----------+
| col1 | INT | TRUE | | | |
| foo | INT | TRUE | | | |
+------+------+------+-----+--------+-----------+
2 rows in set
-- Modify a column
Flink SQL> ALTER TABLE test01 MODIFY col1 STRING;
[INFO] Execute statement succeeded.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] [docs] Flink can now add/drop/modify columns [iceberg]
Posted by "rmoff (via GitHub)" <gi...@apache.org>.
rmoff commented on code in PR #13617:
URL: https://github.com/apache/iceberg/pull/13617#discussion_r2222355767
##########
docs/docs/flink.md:
##########
@@ -401,4 +401,3 @@ There are some features that are do not yet supported in the current Flink Icebe
* Don't support creating iceberg table with hidden partitioning. [Discussion](http://mail-archives.apache.org/mod_mbox/flink-dev/202008.mbox/%3cCABi+2jQCo3MsOa4+ywaxV5J-Z8TGKNZDX-pQLYB-dG+dVUMiMw@mail.gmail.com%3e) in flink mail list.
* Don't support creating iceberg table with computed column.
* Don't support creating iceberg table with watermark.
Review Comment:
shall we remove that in this PR too? or open a new one?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] [docs] Flink can now add/drop/modify columns [iceberg]
Posted by "pvary (via GitHub)" <gi...@apache.org>.
pvary merged PR #13617:
URL: https://github.com/apache/iceberg/pull/13617
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] [docs] Flink can now add/drop/modify columns [iceberg]
Posted by "mxm (via GitHub)" <gi...@apache.org>.
mxm commented on code in PR #13617:
URL: https://github.com/apache/iceberg/pull/13617#discussion_r2221879410
##########
docs/docs/flink.md:
##########
@@ -401,4 +401,3 @@ There are some features that are do not yet supported in the current Flink Icebe
* Don't support creating iceberg table with hidden partitioning. [Discussion](http://mail-archives.apache.org/mod_mbox/flink-dev/202008.mbox/%3cCABi+2jQCo3MsOa4+ywaxV5J-Z8TGKNZDX-pQLYB-dG+dVUMiMw@mail.gmail.com%3e) in flink mail list.
* Don't support creating iceberg table with computed column.
* Don't support creating iceberg table with watermark.
-* Don't support adding columns, removing columns, renaming columns, changing columns. [FLINK-19062](https://issues.apache.org/jira/browse/FLINK-19062) is tracking this.
Review Comment:
Based on the [JIRA](https://issues.apache.org/jira/browse/FLINK-21634), this is only available from Flink 2.1.0 onwards. Maybe add a note about the Flink version which resolved this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] [docs] Flink can now add/drop/modify columns [iceberg]
Posted by "rmoff (via GitHub)" <gi...@apache.org>.
rmoff commented on code in PR #13617:
URL: https://github.com/apache/iceberg/pull/13617#discussion_r2222197548
##########
docs/docs/flink.md:
##########
@@ -401,4 +401,3 @@ There are some features that are do not yet supported in the current Flink Icebe
* Don't support creating iceberg table with hidden partitioning. [Discussion](http://mail-archives.apache.org/mod_mbox/flink-dev/202008.mbox/%3cCABi+2jQCo3MsOa4+ywaxV5J-Z8TGKNZDX-pQLYB-dG+dVUMiMw@mail.gmail.com%3e) in flink mail list.
* Don't support creating iceberg table with computed column.
* Don't support creating iceberg table with watermark.
-* Don't support adding columns, removing columns, renaming columns, changing columns. [FLINK-19062](https://issues.apache.org/jira/browse/FLINK-19062) is tracking this.
Review Comment:
I think they were added in 1.17—there's just one subtask that remains open for 2.1 (and not a significant one that would negate this PR's change)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] [docs] Flink can now add/drop/modify columns [iceberg]
Posted by "pvary (via GitHub)" <gi...@apache.org>.
pvary commented on PR #13617:
URL: https://github.com/apache/iceberg/pull/13617#issuecomment-3158349969
Merged to main.
Thanks for the PR @rmoff and @mxm for the review!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] [docs] Flink can now add/drop/modify columns [iceberg]
Posted by "pvary (via GitHub)" <gi...@apache.org>.
pvary commented on code in PR #13617:
URL: https://github.com/apache/iceberg/pull/13617#discussion_r2228167826
##########
docs/docs/flink.md:
##########
@@ -401,4 +401,3 @@ There are some features that are do not yet supported in the current Flink Icebe
* Don't support creating iceberg table with hidden partitioning. [Discussion](http://mail-archives.apache.org/mod_mbox/flink-dev/202008.mbox/%3cCABi+2jQCo3MsOa4+ywaxV5J-Z8TGKNZDX-pQLYB-dG+dVUMiMw@mail.gmail.com%3e) in flink mail list.
* Don't support creating iceberg table with computed column.
* Don't support creating iceberg table with watermark.
Review Comment:
Yes @rmoff, those are the 2 last missing features mentioned here.
Could you please update the doc accordingly?
Thanks,
Peter
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] [docs] Flink can now add/drop/modify columns [iceberg]
Posted by "mxm (via GitHub)" <gi...@apache.org>.
mxm commented on code in PR #13617:
URL: https://github.com/apache/iceberg/pull/13617#discussion_r2228588937
##########
docs/docs/flink.md:
##########
@@ -401,4 +401,3 @@ There are some features that are do not yet supported in the current Flink Icebe
* Don't support creating iceberg table with hidden partitioning. [Discussion](http://mail-archives.apache.org/mod_mbox/flink-dev/202008.mbox/%3cCABi+2jQCo3MsOa4+ywaxV5J-Z8TGKNZDX-pQLYB-dG+dVUMiMw@mail.gmail.com%3e) in flink mail list.
* Don't support creating iceberg table with computed column.
* Don't support creating iceberg table with watermark.
Review Comment:
>is that right?
Correct.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] [docs] Flink can now add/drop/modify columns [iceberg]
Posted by "rmoff (via GitHub)" <gi...@apache.org>.
rmoff commented on code in PR #13617:
URL: https://github.com/apache/iceberg/pull/13617#discussion_r2254835272
##########
docs/docs/flink.md:
##########
@@ -401,4 +401,3 @@ There are some features that are do not yet supported in the current Flink Icebe
* Don't support creating iceberg table with hidden partitioning. [Discussion](http://mail-archives.apache.org/mod_mbox/flink-dev/202008.mbox/%3cCABi+2jQCo3MsOa4+ywaxV5J-Z8TGKNZDX-pQLYB-dG+dVUMiMw@mail.gmail.com%3e) in flink mail list.
* Don't support creating iceberg table with computed column.
* Don't support creating iceberg table with watermark.
Review Comment:
Fixed d60a9a8c94d9bf42c27a4545dd9ed50c36e617e2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] [docs] Flink can now add/drop/modify columns [iceberg]
Posted by "rmoff (via GitHub)" <gi...@apache.org>.
rmoff commented on code in PR #13617:
URL: https://github.com/apache/iceberg/pull/13617#discussion_r2222630232
##########
docs/docs/flink.md:
##########
@@ -401,4 +401,3 @@ There are some features that are do not yet supported in the current Flink Icebe
* Don't support creating iceberg table with hidden partitioning. [Discussion](http://mail-archives.apache.org/mod_mbox/flink-dev/202008.mbox/%3cCABi+2jQCo3MsOa4+ywaxV5J-Z8TGKNZDX-pQLYB-dG+dVUMiMw@mail.gmail.com%3e) in flink mail list.
* Don't support creating iceberg table with computed column.
* Don't support creating iceberg table with watermark.
Review Comment:
yes, makes sense. So the two remaining features to list as not implemented are:
* Don't support creating iceberg table with hidden partitioning. [Discussion](http://mail-archives.apache.org/mod_mbox/flink-dev/202008.mbox/%3cCABi+2jQCo3MsOa4+ywaxV5J-Z8TGKNZDX-pQLYB-dG+dVUMiMw@mail.gmail.com%3e) in flink mail list.
* Don't support creating iceberg table with computed column.
is that right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] [docs] Flink can now add/drop/modify columns [iceberg]
Posted by "mxm (via GitHub)" <gi...@apache.org>.
mxm commented on code in PR #13617:
URL: https://github.com/apache/iceberg/pull/13617#discussion_r2222323146
##########
docs/docs/flink.md:
##########
@@ -401,4 +401,3 @@ There are some features that are do not yet supported in the current Flink Icebe
* Don't support creating iceberg table with hidden partitioning. [Discussion](http://mail-archives.apache.org/mod_mbox/flink-dev/202008.mbox/%3cCABi+2jQCo3MsOa4+ywaxV5J-Z8TGKNZDX-pQLYB-dG+dVUMiMw@mail.gmail.com%3e) in flink mail list.
* Don't support creating iceberg table with computed column.
* Don't support creating iceberg table with watermark.
Review Comment:
Side note: This is also not true anymore: https://github.com/apache/iceberg/pull/12191
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] [docs] Flink can now add/drop/modify columns [iceberg]
Posted by "rmoff (via GitHub)" <gi...@apache.org>.
rmoff commented on code in PR #13617:
URL: https://github.com/apache/iceberg/pull/13617#discussion_r2254836578
##########
docs/docs/flink.md:
##########
@@ -401,4 +401,3 @@ There are some features that are do not yet supported in the current Flink Icebe
* Don't support creating iceberg table with hidden partitioning. [Discussion](http://mail-archives.apache.org/mod_mbox/flink-dev/202008.mbox/%3cCABi+2jQCo3MsOa4+ywaxV5J-Z8TGKNZDX-pQLYB-dG+dVUMiMw@mail.gmail.com%3e) in flink mail list.
* Don't support creating iceberg table with computed column.
* Don't support creating iceberg table with watermark.
Review Comment:
@pvary could you review/merge please? thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] [docs] Flink can now add/drop/modify columns [iceberg]
Posted by "mxm (via GitHub)" <gi...@apache.org>.
mxm commented on code in PR #13617:
URL: https://github.com/apache/iceberg/pull/13617#discussion_r2222416611
##########
docs/docs/flink.md:
##########
@@ -401,4 +401,3 @@ There are some features that are do not yet supported in the current Flink Icebe
* Don't support creating iceberg table with hidden partitioning. [Discussion](http://mail-archives.apache.org/mod_mbox/flink-dev/202008.mbox/%3cCABi+2jQCo3MsOa4+ywaxV5J-Z8TGKNZDX-pQLYB-dG+dVUMiMw@mail.gmail.com%3e) in flink mail list.
* Don't support creating iceberg table with computed column.
* Don't support creating iceberg table with watermark.
Review Comment:
While we're on it, we can simply remove this one as well. What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] [docs] Flink can now add/drop/modify columns [iceberg]
Posted by "mxm (via GitHub)" <gi...@apache.org>.
mxm commented on code in PR #13617:
URL: https://github.com/apache/iceberg/pull/13617#discussion_r2222322228
##########
docs/docs/flink.md:
##########
@@ -401,4 +401,3 @@ There are some features that are do not yet supported in the current Flink Icebe
* Don't support creating iceberg table with hidden partitioning. [Discussion](http://mail-archives.apache.org/mod_mbox/flink-dev/202008.mbox/%3cCABi+2jQCo3MsOa4+ywaxV5J-Z8TGKNZDX-pQLYB-dG+dVUMiMw@mail.gmail.com%3e) in flink mail list.
* Don't support creating iceberg table with computed column.
* Don't support creating iceberg table with watermark.
-* Don't support adding columns, removing columns, renaming columns, changing columns. [FLINK-19062](https://issues.apache.org/jira/browse/FLINK-19062) is tracking this.
Review Comment:
True, I've been using those for a while :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org