You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Flink Jira Bot (Jira)" <ji...@apache.org> on 2022/02/10 10:38:00 UTC

[jira] [Updated] (FLINK-2396) Review the datasets of dynamic path and static path in iteration.

     [ https://issues.apache.org/jira/browse/FLINK-2396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Flink Jira Bot updated FLINK-2396:
----------------------------------
      Labels: auto-deprioritized-major auto-deprioritized-minor auto-unassigned  (was: auto-deprioritized-major auto-unassigned stale-minor)
    Priority: Not a Priority  (was: Minor)

This issue was labeled "stale-minor" 7 days ago and has not received any updates so it is being deprioritized. If this ticket is actually Minor, please raise the priority and ask a committer to assign you the issue or revive the public discussion.


> Review the datasets of dynamic path and static path in iteration.
> -----------------------------------------------------------------
>
>                 Key: FLINK-2396
>                 URL: https://issues.apache.org/jira/browse/FLINK-2396
>             Project: Flink
>          Issue Type: Improvement
>          Components: Runtime / Task
>            Reporter: Chengxiang Li
>            Priority: Not a Priority
>              Labels: auto-deprioritized-major, auto-deprioritized-minor, auto-unassigned
>
> Currently Flink would cached dataset in static path as it assumes that dataset stay the same during the iteration, but this assumption does not always be true. Take sampling for example, the iteration data set is something like the weight vector of model and there is another training dataset from which to take a small sample to update the weight vector in each iteration (e.g. Stochastic Gradient Descent), we expect sampled dataset is different in each iteration, but Flink would cache the sampled dataset as it in static path. 
> We should review how Flink identify dynamic path and static path, and support add sampled dataset in above example to dynamic path.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)