You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "Gerrrr (via GitHub)" <gi...@apache.org> on 2023/10/05 01:39:22 UTC
[I] Support deletion in Apache Flink [iceberg]
Gerrrr opened a new issue, #8718:
URL: https://github.com/apache/iceberg/issues/8718
### Feature Request / Improvement
Hello community,
Is `DELETE FROM` support for Apache Flink already in the roadmap? Example:
```
Flink SQL> DELETE FROM word_count WHERE word = 'z';
[ERROR] Could not execute SQL statement. Reason:
java.lang.UnsupportedOperationException: Can't perform delete operation of the table nessie_catalog.db.word_count because the corresponding dynamic table sink has not yet implemented org.apache.flink.table.connector.sink.abilities.SupportsRowLevelDelete.
```
It looks like this functionality already exists at least for Apache Spark - https://iceberg.apache.org/docs/latest/spark-writes/#delete-from.
### Query engine
Flink
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Support deletion in Apache Flink [iceberg]
Posted by "rlokugamage (via GitHub)" <gi...@apache.org>.
rlokugamage commented on issue #8718:
URL: https://github.com/apache/iceberg/issues/8718#issuecomment-3299053221
> [@rlokugamage](https://github.com/rlokugamage): Here is where we state, that update is possible: https://iceberg.apache.org/docs/nightly/flink-writes/#insert-into
>
> The documentation doesn't state how it is done, but it is working as I have described above.
@pvary so if I just wanted to delete some data in a table, I just need a datastream with RowKind.DELETE rows?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Support deletion in Apache Flink [iceberg]
Posted by "rlokugamage (via GitHub)" <gi...@apache.org>.
rlokugamage commented on issue #8718:
URL: https://github.com/apache/iceberg/issues/8718#issuecomment-3300493405
awesome, do you know if it's writing equality or positional deletes?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Support deletion in Apache Flink [iceberg]
Posted by "pvary (via GitHub)" <gi...@apache.org>.
pvary commented on issue #8718:
URL: https://github.com/apache/iceberg/issues/8718#issuecomment-1980627773
If your input stream contains `-D` records, then they will be deleted from the table.
Like:
```java
Row row = Row.withPositions(RowKind.DELETE, 3);
row.setField(0, "hello");
row.setField(1, true);
row.setField(2, 1L);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Support deletion in Apache Flink [iceberg]
Posted by "pvary (via GitHub)" <gi...@apache.org>.
pvary commented on issue #8718:
URL: https://github.com/apache/iceberg/issues/8718#issuecomment-1748095935
Is this for a V2 table?
I have seen deleting rows working using V2 table, Java code with the stream API, but I yet to try out SQL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Support deletion in Apache Flink [iceberg]
Posted by "pvary (via GitHub)" <gi...@apache.org>.
pvary commented on issue #8718:
URL: https://github.com/apache/iceberg/issues/8718#issuecomment-3298964452
@rlokugamage: Here is where we state, that update is possible: https://iceberg.apache.org/docs/nightly/flink-writes/#insert-into
The documentation doesn't state how it is done, but it is working as I have described above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Support deletion in Apache Flink [iceberg]
Posted by "Gerrrr (via GitHub)" <gi...@apache.org>.
Gerrrr commented on issue #8718:
URL: https://github.com/apache/iceberg/issues/8718#issuecomment-1748157735
Yes, this is a V2 table. I added the DDL to the description now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Support deletion in Apache Flink [iceberg]
Posted by "VidakM (via GitHub)" <gi...@apache.org>.
VidakM commented on issue #8718:
URL: https://github.com/apache/iceberg/issues/8718#issuecomment-1917071229
Is `DELTE FROM` still on the roadmap for Flink or could we help out in some way?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Support deletion in Apache Flink [iceberg]
Posted by "rlokugamage (via GitHub)" <gi...@apache.org>.
rlokugamage commented on issue #8718:
URL: https://github.com/apache/iceberg/issues/8718#issuecomment-3293507107
> If your input stream contains `-D` records, then they will be deleted from the table. Like:
>
> Row row = Row.withPositions(RowKind.DELETE, 3);
> row.setField(0, "hello");
> row.setField(1, true);
> row.setField(2, 1L);
@pvary is this still the case? Nothing in Iceberg's or Flink's documentation state that it can do row level deletes. I need to test it out myself still but just needed some confirmation. Are they positional deletes or equality deletes?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Support deletion in Apache Flink [iceberg]
Posted by "AjayChitumalla (via GitHub)" <gi...@apache.org>.
AjayChitumalla commented on issue #8718:
URL: https://github.com/apache/iceberg/issues/8718#issuecomment-1979125075
> Is this for a V2 table? I have seen deleting rows working using V2 table, Java code with the stream API, but I yet to try out SQL.
Can you share a reference for performing row level deletes on Iceberg V2 table using Flink DataStream API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Support deletion in Apache Flink [iceberg]
Posted by "pvary (via GitHub)" <gi...@apache.org>.
pvary commented on issue #8718:
URL: https://github.com/apache/iceberg/issues/8718#issuecomment-3300404774
> > [@rlokugamage](https://github.com/rlokugamage): Here is where we state, that update is possible: https://iceberg.apache.org/docs/nightly/flink-writes/#insert-into
> > The documentation doesn't state how it is done, but it is working as I have described above.
>
> [@pvary](https://github.com/pvary) so if I just wanted to delete some data in a table, I just need a datastream with RowKind.DELETE rows?
Exactly
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Support deletion in Apache Flink [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #8718:
URL: https://github.com/apache/iceberg/issues/8718#issuecomment-2365374220
This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org