[GitHub] [iceberg] zhangdove commented on issue #2319: Caching Tables in SparkCatalog via CachingCatalog by default leads to stale data

[GitBox <gi...@apache.org>]

zhangdove commented on issue #2319:
URL: https://github.com/apache/iceberg/issues/2319#issuecomment-796413642


   I also agree that the default 'cache-enable' is set to false and it is up to the user to turn this parameter on or not.
   
   In particular, some long lived Spark Sessions may have exceptions due to default caching. There are two kinds of situations that we encounter:
   
   1. Asynchronous job clears old snapshots and data files (the higher the frequency of triggering, the more likely it is to occur)
   2. Asynchronous job cleaning orphan files will clear older metadata.json (write.metadata.previous-versions-max default value is 100)


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org