[jira] [Commented] (FLINK-7771) Make the operator state queryable

["Kostas Kloudas (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-7771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16332121#comment-16332121 ] 

Kostas Kloudas commented on FLINK-7771:
---------------------------------------

[~skonto] Informing an external system about state changes is not a general solution. This implies that the "ground truth" for some internal Flink progress metrics will be in an external storage system. In addition, in its generalized form where you can ask the op state at a specific point in time, it could also require changes to the state backend logic about when and how to garbage collect files belonging to old checkpoints.

 

Given this, I think that 1) it is not a clear solution, and 2) there is a lot of work required which will be redundant as soon as proper support for some other state management issues comes. 

 

Could you elaborate a bit on why exactly you need to make the operator state queryable? Could Broadcast State also fit the bill (have a stream whose elements are sent to all downstream tasks connected with another stream with fresh data)? I am asking because there is an open PR for this, and hopefully it will get merged soon.

The PR is here (although still under development and without much documentation) : https://github.com/apache/flink/pull/5230

 

> Make the operator state queryable
> ---------------------------------
>
>                 Key: FLINK-7771
>                 URL: https://issues.apache.org/jira/browse/FLINK-7771
>             Project: Flink
>          Issue Type: Improvement
>          Components: Queryable State
>    Affects Versions: 1.4.0
>            Reporter: Kostas Kloudas
>            Assignee: Kostas Kloudas
>            Priority: Major
>
> There seem to be some requests for making the operator (non-keyed) state queryable. This means that the user will specify the *uuid* of the operator and the *taskId*, and he will be able to access the state that corresponds to that operator and for that specific task.
> This issue will serve to document the discussion on the topic, so that everybody can participate.
> I also link [~till.rohrmann] and [~skonto] as he also mentioned that this feature could be helpful.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)