You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Roman Khachatryan (Jira)" <ji...@apache.org> on 2021/06/17 17:15:00 UTC
[jira] [Updated] (FLINK-21804) Create and wire changelog writer
with backend
[ https://issues.apache.org/jira/browse/FLINK-21804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Roman Khachatryan updated FLINK-21804:
--------------------------------------
Attachment: Changelog Backend _ writer loading.png
> Create and wire changelog writer with backend
> ---------------------------------------------
>
> Key: FLINK-21804
> URL: https://issues.apache.org/jira/browse/FLINK-21804
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / State Backends
> Reporter: Roman Khachatryan
> Priority: Major
> Fix For: 1.14.0
>
> Attachments: Changelog Backend _ writer loading.png
>
>
> As of FLINK-21355, changelog writer factory is created inside the backend while creating keyed backend.
>
> Instead, it should be obtained from the environment (env.getTaskStateManager.get...). Creation should be done closer to TaskExecutor/Task creation, so that PluginManager is accessible and grouping across subtasks is possible.
>
> The previous approach (removed from FLINK-21355 PR) was to use StateBackendLoader, pass the created factory to backend, and then to keyed backend. The issue is that it is also used in JM and other environments when no state is actually logged. It's also difficult to use PluginManager there.
>
> Motivation to use PluginManager:
> 1. allow different changelog implementations
> 2. (possibly) allow using changelog in different places
> However, this would require moving backend loading closer to TaskManagerRunner where other pluginized components are initialized.
> Probably, this should be done together with pluginizing state backends.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)