[jira] [Comment Edited] (PHOENIX-6141) Ensure consistency between SYSTEM.CATALOG and SYSTEM.CHILD_LINK

["Chinmay Kulkarni (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PHOENIX-6141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197882#comment-17197882 ] 

Chinmay Kulkarni edited comment on PHOENIX-6141 at 9/17/20, 6:14 PM:
---------------------------------------------------------------------

FWIW it is actually pretty easy to create an orphan linking row in SYSTEM.CHILD_LINK. You can create 2 tables (or views) T1, T2 and try to create the same named view on top of both. Though the second view creation will fail with a TableAlreadyExistsException, the RPC to insert the parent→child link for this view would have already gone through and is now an orphan link. 

Even if you issue the same create view statement (on the same parent) twice and the second request fails, the parent->child link will be overwritten and its time stamp changes.


was (Author: ckulkarni):
FWIW it is actually pretty easy to create an orphan linking row in SYSTEM.CHILD_LINK. You can create 2 tables (or views) T1, T2 and try to create the same named view on top of both. Though the second view creation will fail with a TableAlreadyExistsException, the RPC to insert the parent→child link for this view would have already gone through and is now an orphan link. 

Even if you issue the same create view statement for the same view twice and the second request fails, the parent->child link will be overwritten and its time stamp changes.

> Ensure consistency between SYSTEM.CATALOG and SYSTEM.CHILD_LINK
> ---------------------------------------------------------------
>
>                 Key: PHOENIX-6141
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-6141
>             Project: Phoenix
>          Issue Type: Improvement
>    Affects Versions: 5.0.0, 4.15.0
>            Reporter: Chinmay Kulkarni
>            Priority: Major
>             Fix For: 4.17.0
>
>
> Before 4.15, "CREATE/DROP VIEW" was an atomic operation since we were issuing batch mutations on just the 1 SYSTEM.CATALOG region. In 4.15 we introduced SYSTEM.CHILD_LINK to store the parent->child links and so a CREATE VIEW is no longer atomic since it consists of 2 separate RPCs  (1 to SYSTEM.CHILD_LINK to add the linking row and another to SYSTEM.CATALOG to write metadata for the new view). 
> If the second RPC i.e. the RPC to write metadata to SYSTEM.CATALOG fails after the 1st RPC has already gone through, there will be an inconsistency between both metadata tables. We will see orphan parent->child linking rows in SYSTEM.CHILD_LINK in this case. This can cause the following issues:
> # ALTER TABLE calls on the base table will fail
> # DROP TABLE without CASCADE will fail
> # The upgrade path has calls like UpgradeUtil.upgradeTable() which will fail
> # Any metadata consistency checks can be thrown off
> # Unnecessary extra storage of orphan links
> The first 3 issues happen because we wrongly deduce that a base table has child views due to the orphan linking rows.
> This Jira aims at trying to come up with a way to make mutations among SYSTEM.CATALOG and SYSTEM.CHILD_LINK an atomic transaction. We can use a 2-phase commit approach like in global indexing or also potentially explore using a transaction manager. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)