[jira] [Commented] (PHOENIX-6685) Change Data Capture - Populate Table / Topic Mappings

["Geoffrey Jacoby (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PHOENIX-6685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17522540#comment-17522540 ] 

Geoffrey Jacoby commented on PHOENIX-6685:
------------------------------------------

[~bharathv] [~jain.ankit] - do you have any concerns with switching to use a field on System.Catalog to store the table / topic mapping? Earlier in the project I think we were trying to keep CDC details out of syscat, but now that we're already storing external schema id as a 1:1 mapping with a table or view there that doesn't seem to be a design goal anymore. 

> Change Data Capture - Populate Table / Topic Mappings 
> ------------------------------------------------------
>
>                 Key: PHOENIX-6685
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-6685
>             Project: Phoenix
>          Issue Type: Sub-task
>            Reporter: Geoffrey Jacoby
>            Assignee: Geoffrey Jacoby
>            Priority: Major
>             Fix For: 5.2.0
>
>
> When creating a new table with CHANGE_DETECTION_ENABLED, or when altering an existing table to set this property to true, we should persist a mapping between the table being enabled, and the message broker topic which will carry its change messages. 
> This will allow both producers and consumers to use Phoenix metadata APIs to figure out which topic to produce or consume from. 
> In the CDC prototype my colleagues and I have been working with, this has been using a distinct system table we've manually populated. However, we may want to consider using System.Catalog linking rows instead, to make things more atomic. If table and topic will always be 1:1, we can also consider just adding an optional field to SYSTEM.CATALOG that views and tables can populate.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)