[GitHub] [iceberg] aokolnychyi commented on pull request #7194: Core, AWS: Auto optimize table using post commit notifications

["aokolnychyi (via GitHub)" <gi...@apache.org>]

aokolnychyi commented on PR #7194:
URL: https://github.com/apache/iceberg/pull/7194#issuecomment-1496709901

   There can be many different ways to handle table management. By adding specific table properties we are forcing a particular implementation. I don't think it is a good idea. In my view, it is OK for vendors to provide their own implementations and Iceberg as a library should allow those implementation to plug in. That means if the Iceberg repo needs to adapt to inject extra metrics reporters during writes or if we need another AutoOptimize API to make this happen, we should do it. However, this seems too specific to AWS to live in the Iceberg repo directly. When we created actions, the assumption was that we would share common table management blocks but we won't force how those blocks are used/triggered.
   
   We could discuss some form of auto optimize API that would kick in after writes (sync or async) and use the same cluster and resources. That could make sense but starting an async compaction job on EMR seems too specific to live here directly.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org