[GitHub] [iceberg] kbendick edited a comment on pull request #3543: Add configurable cache expiry to caching catalog

[GitBox <gi...@apache.org>]

kbendick edited a comment on pull request #3543:
URL: https://github.com/apache/iceberg/pull/3543#issuecomment-985818457


   > Just to resonate with what Russell said, I have also heard a few customers got confused about the caching behaviors, and not enabling caching directly would likely be a more straight-forward solution for users, compared to have a 30 seconds expiration that we have to then explain potential behaviors of query based on this expiration time.
   > 
   > Overall I think it's worth adding the feature, Trino and Presto also has something similar like `hive.metastore-refresh-interval`. I would personally just do not set a default for that config, given it is an optional config anyway.
   > 
   > Given the fact that the PR is also getting quite messy with all the changes, I would say let's first add the `cache-enabled` as a catalog property with `true` as default, which is the default behavior today. Then we can add time-based expiration and review it separately in more details.
   
   So you think I should open a new PR that just has a flag for `expiration-enabled`, @jackye1995?
   
   There is a `cache-enabled` flag already.
   
   It's also worth noting that Spark has added a similar feature, where it stops caching table execution plans after 30s or so via a similar strategy.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org