[jira] [Created] (IMPALA-9791) Support table validWriteIdList in getPartitionCatalogInfoRequest

["Vihang Karajgaonkar (Jira)" <ji...@apache.org>]

Vihang Karajgaonkar created IMPALA-9791:
-------------------------------------------

             Summary: Support table validWriteIdList in getPartitionCatalogInfoRequest
                 Key: IMPALA-9791
                 URL: https://issues.apache.org/jira/browse/IMPALA-9791
             Project: IMPALA
          Issue Type: Improvement
            Reporter: Vihang Karajgaonkar
            Assignee: Vihang Karajgaonkar


When transactional tables are being queried, the coordinator (or any other Catalog client) can optionally provide a ValidWriteIdList of the table. In such case, catalog can return the metadata which is consistent with the given ValidWriteIdList. There are the following 3 possibilities:

1. Client provided ValidWriteIdList is more recent.
In this case, catalog should reload the table then send the metadata consistent with the provided writeIdList.

2. Client ValidWriteIdList is same.
Catalog can return the cached metadata directly.

3. ClientValidWriteIdList is stale with respect to the one in catalog.
In this case, catalog can attempt to return metadata which is consistent with respect to client's view of the writeIdList and return accordingly. Note that in case 1, it is possible that after reload, catalog moves ahead of the client's writeIdList and hence this becomes a sub-case of 1.

Having such an enhancement to the API can help support consistent read support for ACID tables (see IMPALA-8788)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)