You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by "Yi Pan (Data Infrastructure) (JIRA)" <ji...@apache.org> on 2018/09/06 23:24:00 UTC
[jira] [Assigned] (SAMZA-1815) Use TableDescriptors to generate
configurations during planning phase
[ https://issues.apache.org/jira/browse/SAMZA-1815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yi Pan (Data Infrastructure) reassigned SAMZA-1815:
---------------------------------------------------
Assignee: Yi Pan (Data Infrastructure)
> Use TableDescriptors to generate configurations during planning phase
> ---------------------------------------------------------------------
>
> Key: SAMZA-1815
> URL: https://issues.apache.org/jira/browse/SAMZA-1815
> Project: Samza
> Issue Type: Improvement
> Reporter: Bharath Kumarasubramanian
> Assignee: Yi Pan (Data Infrastructure)
> Priority: Major
>
> Currently, the pattern followed to materialize table descriptors into configuration is as follows.
> # User application provides a TableDescriptor
> # Samza constructs a TableSpec from the table descriptor
> # In the planning phase, Samza uses the table provider factory from the table spec to create a concrete TableProvider instance.
> # The concrete instance is used to generate the actual configurations for the table and shoves them into configs.
> With [https://cwiki.apache.org/confluence/display/SAMZA/SEP-14%3A+System+and+Stream+Descriptors], we can follow a similar pattern to use the TableDescriptors directly to materialize table related configurations. It buys as the following
> * Consistency with the existing pattern
> * Removes the need of intermediate TableSpec
> * Eliminates the dual responsibility of the TableProvider interface which currently exposes methods that partly used in planning phase and some used during runtime.
> * Removes intermediate internal configurations that used as a placeholder to pass information to provider from descriptor such as changelog stream, replication factor etc.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)