[GitHub] [iceberg] Reo-LEI commented on issue #3941: [Feature Request] Support for change data capture

[GitBox <gi...@apache.org>]

Reo-LEI commented on issue #3941:
URL: https://github.com/apache/iceberg/issues/3941#issuecomment-1038873547


   > @flyrain We are looking into two possible approaches here.
   > 1. Materialised where, like DeltaLake Change Data Feed that is generated in the write path.
   > 2. Change Data Feed is evaluated at the read path by consumers.
   
   I drafted a [design document](https://docs.google.com/document/d/1zEpNYcA5Tf5ysdoj3jO425A1QRI-3OMb_Fy8EG_9DD4/edit?usp=sharing) for case 1 base on write-time logging. 
   
   But as @flyrain mentioned in email, the approach based on write-time logging would require changing the table spec and changing all engines to fit the new table spec. Therefore, I would tend to take the approach of @flyrain and try to integrate it into flink. 
   
   Just paste my approach here so we can find it later when we need it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org