Proposal : Last Window Store for Output Operators

[Sandesh Hegde <sa...@datatorrent.com>]

Hi All,

Our current design pattern for using Window Data Manager is to purge the
Window Data in the committed window call back.

In the case of Output operators, all we care is the Last (/window)
snapshot, that is because we have already written the tuples before that
snapshot so those can be safely ignored. So there is no need to keep
multiple checkpoints around.

This behaviour can be achieved in the WindowDataManager but having the
separate interface makes it easy for the developers.

So the proposed interface, for the pattern mentioned, is as below,

public interface LastStoredState {
        long getWindowId();
        Map<Integer, Object>  load()
        void save(<Data>, windowId, ) //Overwrite the previous window.
        void delete()   // Cleanup
}

Please let me know your thoughts.

Thanks