[jira] [Commented] (SAMZA-1270) Pull out MetricsReporter lifecycle from Container

["Navina Ramesh (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SAMZA-1270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003842#comment-16003842 ] 

Navina Ramesh commented on SAMZA-1270:
--------------------------------------

This issue is much harder than it seems. It feels like a chicken-and-egg problem because each instance of {{MetricsReporter}} is tied a particular instance of {{SamzaContainer}}. In order to make this reporter instance tied to the lifeycle of the {{StreamProcessor}}, couple of changes need to be made:
1. Even after SEP-1, it is not clear why {{JobCoordinator}} has to be the source of truth for the _processorId_. The reality is that even in Zk based JC, the _processorId_ is tied only to its executing environment and not ZK generated Ids. There is no reason for JobCoordinator to be the source of truth of the identifier. 
2. Since _processorId_ is tied to the executing environment, it should be provided to the ??StreamProcessor?? by the ??ApplicationRunner?? abstraction.

Once #1 and #2 are resolved, it should be straight-forward to manage the lifecycle of the Metric Reporters within StreamProcessor and share it between SamzaContainer and JobCoordinator. 


> Pull out MetricsReporter lifecycle from Container
> -------------------------------------------------
>
>                 Key: SAMZA-1270
>                 URL: https://issues.apache.org/jira/browse/SAMZA-1270
>             Project: Samza
>          Issue Type: Improvement
>            Reporter: Navina Ramesh
>            Assignee: Navina Ramesh
>             Fix For: 0.14.0
>
>
> Currently, the lifecycle of many components, including the {{MetricsReporter}} is tied to the container lifecycle. Due to this, StreamProcessor/JobCoordinator will need to manage their own metrics registry and reporters. 
> In order to unify the stack, the ideal place for metricsReporter lifecycle will be in the user-facing apis - LocalContainerRunner & LocalApplicationRunner. Every component will simply register its metrics to the reporter dynamically. Iiuc, the {{MetricsReporter}} can implement a {{ReadableMetricsRegistryListener}} interface to register metrics dynamically. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)