Re: [PR] [core] Invalidate partition cache when dropping a database [paimon]

["JingsongLi (via GitHub)" <gi...@apache.org>]

JingsongLi commented on PR #10259:
URL: https://github.com/apache/paimon/pull/10259#issuecomment-5937052381

   [P2] Preserve database-wide invalidation of already cached tables
   
   At `CachingCatalog.java:207–216`, replacing the local `tableCache` key enumeration with only `listTables(name)` misses cached tables that another catalog/client has already deleted. Reproduction with the real filesystem catalog: cache `db.t` with an INT column; delete `t` through the underlying catalog; call the caching catalog's `dropDatabase("db", false, true)`; recreate `db.t` with a STRING column. The drop succeeds, but `listTables` was empty and the cached table survives. `getTable` returns the old INT schema, and an actual write of a STRING row fails with `ClassCastException`. The same problem occurs when the other client deletes the whole database and the local call uses `ignoreIfNotExists=true` (the caught exception leaves `tables` empty). The baseline clears the local table entry and both writes succeed.
   
   Please retain the database-wide cleanup of local cache keys, alongside the partition-cache invalidation added here—for example, invalidate the union of the enumerated tables and locally cached identifiers after a successful drop, including identifiers belonging to absent tables/databases. Ideally clear the corresponding partition entries by database as well. The normal cascade/recreate case does verify the intended partition-cache fix; this report specifically concerns the newly lost table-cache invalidation.
   
   Validation: all 60 `CachingCatalogTest` tests passed with normal Maven checks. I additionally ran actual create/write/cache/drop/recreate/write/read scenarios for normal cascade, an externally deleted table, and an externally deleted database, with the baseline `CachingCatalog` as a control.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org