[I] Notification UPDATE for an absent table can leave older metadata installed when two notifications race to create it [polaris]

["harshitaajoshi (via GitHub)" <gi...@apache.org>]

harshitaajoshi opened a new issue, #5621:
URL: https://github.com/apache/polaris/issues/5621

   Follow-up to #4658 and #5222.
   
   #5222 adds a bounded retry for notification `UPDATE`s on an existing table. It does not cover the case where the table does not exist yet.
   
   When two `UPDATE` notifications arrive for a table that is not registered, both take the `existingLocation == null` branch in `LocalIcebergCatalog.sendNotificationForTableLike` and try to create the table. If the notification with the lower timestamp wins the create, the one with the higher timestamp fails with `AlreadyExistsException`, and the older metadata location stays installed.
   
   ### Expected behavior
   
   The notification with the newest timestamp should end up installed, or the losing request should fail in a way that lets the sender retry and have its newer timestamp admitted.
   
   ### Possible direction
   
   When the create loses the name race, reload the entity that won and fall through to the timestamp-checked update path instead of surfacing `AlreadyExistsException`.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Notification UPDATE for an absent table can leave older metadata installed when two notifications race to create it [polaris]

["sush7262 (via GitHub)" <gi...@apache.org>]

sush7262 commented on issue #5621:
URL: https://github.com/apache/polaris/issues/5621#issuecomment-6145627939

   Hi @harshitaajoshi, I'd like to work on this issue!
   
   Proposed Approach:
   1. In `LocalIcebergCatalog.sendNotificationForTableLike`, when an `UPDATE` notification loses the create race and throws `AlreadyExistsException`, catch it instead of surfacing it immediately.
   2. Reload the newly created entity via `resolvedEntityView.getPassthroughResolvedPath`.
   3. If the incoming notification has a newer timestamp, fall through to `updateTableLike` to apply the latest metadata; otherwise, reject it as out-of-order.
   4. Add unit tests in `AbstractLocalIcebergCatalogTest` to cover racing notifications on absent tables.
   
   Could you please assign this issue to me? Thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org