You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "luoyuxia (via GitHub)" <gi...@apache.org> on 2023/04/10 02:04:48 UTC
[GitHub] [iceberg] luoyuxia opened a new issue, #7311: Support Delete & Update iceberg table with Flink
luoyuxia opened a new issue, #7311:
URL: https://github.com/apache/iceberg/issues/7311
### Feature Request / Improvement
In #7254, Iceberg supports Flink 1.17. In Flink 1.17, Flink introuces [delete & update api](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=235838061) to enable external connector to delete & update data.
I think we can make Flink support delete & update iceberg data which will be useful to help user manage iceberg by Flink.
If someone may want to take it, I'm glad to help review as the designer of Flink delete & update api.
If not, I'll be happy to take it.
### Query engine
Flink
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] clownxc commented on issue #7311: Support Delete & Update iceberg table with Flink
Posted by "clownxc (via GitHub)" <gi...@apache.org>.
clownxc commented on issue #7311:
URL: https://github.com/apache/iceberg/issues/7311#issuecomment-1503538680
hi, I would like to give a try on this, can I take this ticket?I will try my best to complete it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] linyanghao commented on issue #7311: Support Delete & Update iceberg table with Flink
Posted by "linyanghao (via GitHub)" <gi...@apache.org>.
linyanghao commented on issue #7311:
URL: https://github.com/apache/iceberg/issues/7311#issuecomment-1556768161
> Hi, just for reference, Pamion has integrated [UPDATE](https://github.com/apache/incubator-paimon/pull/1010) & [DELETE](https://github.com/apache/incubator-paimon/issues/786) for flink-1.17. Also in previous investagtion, IIUC, to implement DELETE & UPDATE for iceberg, we may need to finish #7110 first since Iceberg will use the metadata columns, file_path, file_pos to delete data.
The meta columns are required for UPDATE & DELETE on tables without pk. But for tables with pk, we can UPDATE & DELETE using the pk, which does not rely on https://github.com/apache/iceberg/issues/7110.
In fact, UPDATE & DELETE in Paimon also only supports tables with pk.
IMHO, we can first support tables with pk. While support for tables without pk can be added later, after https://github.com/apache/iceberg/issues/7110 is finished.
@luoyuxia @clownxc What are your thoughts on this? Thanks!
cc @hililiwei @stevenzwu @chenjunjiedada
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] luoyuxia commented on issue #7311: Support Delete & Update iceberg table with Flink
Posted by "luoyuxia (via GitHub)" <gi...@apache.org>.
luoyuxia commented on issue #7311:
URL: https://github.com/apache/iceberg/issues/7311#issuecomment-1539316414
@clownxc Sure, feel free to take it. I'm glad to help review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] clownxc commented on issue #7311: Support Delete & Update iceberg table with Flink
Posted by "clownxc (via GitHub)" <gi...@apache.org>.
clownxc commented on issue #7311:
URL: https://github.com/apache/iceberg/issues/7311#issuecomment-1553026533
> Hi, just for reference, Pamion has integrated [UPDATE](https://github.com/apache/incubator-paimon/pull/1010) & [DELETE](https://github.com/apache/incubator-paimon/issues/786) for flink-1.17. Also in previous investagtion, IIUC, to implement DELETE & UPDATE for iceberg, we may need to finish #7110 first since Iceberg will use the metadata columns, file_path, file_pos to delete data.
i see, thank you very much for your guidance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] clownxc commented on issue #7311: Support Delete & Update iceberg table with Flink
Posted by "clownxc (via GitHub)" <gi...@apache.org>.
clownxc commented on issue #7311:
URL: https://github.com/apache/iceberg/issues/7311#issuecomment-1551042947
> Hi @clownxc . Is there any progress on this? Thanks!
I'm working on this task, there will be progress recently😊
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Support Delete & Update iceberg table with Flink [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #7311:
URL: https://github.com/apache/iceberg/issues/7311#issuecomment-1828847271
This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] luoyuxia commented on issue #7311: Support Delete & Update iceberg table with Flink
Posted by "luoyuxia (via GitHub)" <gi...@apache.org>.
luoyuxia commented on issue #7311:
URL: https://github.com/apache/iceberg/issues/7311#issuecomment-1552293691
Hi, just for reference, Pamion has integrated [UPDATE](https://github.com/apache/incubator-paimon/pull/1010) & [DELETE](https://github.com/apache/incubator-paimon/issues/786) for flink-1.17.
Also in previous investagtion, IIUC, to implement DELETE & UPDATE for iceberg, we may need to finish #7110 first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Support Delete & Update iceberg table with Flink [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #7311:
URL: https://github.com/apache/iceberg/issues/7311#issuecomment-1854885763
This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] luoyuxia commented on issue #7311: Support Delete & Update iceberg table with Flink
Posted by "luoyuxia (via GitHub)" <gi...@apache.org>.
luoyuxia commented on issue #7311:
URL: https://github.com/apache/iceberg/issues/7311#issuecomment-1556786949
> > Hi, just for reference, Pamion has integrated [UPDATE](https://github.com/apache/incubator-paimon/pull/1010) & [DELETE](https://github.com/apache/incubator-paimon/issues/786) for flink-1.17. Also in previous investagtion, IIUC, to implement DELETE & UPDATE for iceberg, we may need to finish #7110 first since Iceberg will use the metadata columns, file_path, file_pos to delete data.
>
> The meta columns are required for UPDATE & DELETE on tables without pk. But for tables with pk, we can UPDATE & DELETE using the pk, which does not rely on #7110. In fact, UPDATE & DELETE in Paimon also only supports tables with pk.
>
> IMHO, we can first support tables with pk. While support for tables without pk can be added later, after #7110 is finished. @luoyuxia @clownxc What are your thoughts on this? Thanks!
>
> cc @hililiwei @stevenzwu @chenjunjiedada
Sounds good to me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] legendtkl commented on issue #7311: Support Delete & Update iceberg table with Flink
Posted by "legendtkl (via GitHub)" <gi...@apache.org>.
legendtkl commented on issue #7311:
URL: https://github.com/apache/iceberg/issues/7311#issuecomment-1539357189
+1 for this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] linyanghao commented on issue #7311: Support Delete & Update iceberg table with Flink
Posted by "linyanghao (via GitHub)" <gi...@apache.org>.
linyanghao commented on issue #7311:
URL: https://github.com/apache/iceberg/issues/7311#issuecomment-1547464630
Hi @clownxc . Is there any progress on this? Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Support Delete & Update iceberg table with Flink [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #7311: Support Delete & Update iceberg table with Flink
URL: https://github.com/apache/iceberg/issues/7311
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org