Re: Re: [VOTE] FLIP-173: Support DAG of algorithms

[Dong Lin <li...@gmail.com>]

Hi everyone,

We (Jiangjie Qin, Yun Gao, Zhipeng Zhang, Weibo Zhao, Yunfeng Zhou and
myself) would like to make the following minor API change to this FLIP.

We will update Stage's Java doc to say the following, which changed the
load(..) first parameter type from StreamExecutionEnvironment to
StreamTableEnvironment.
The reason is explained in F
<https://issues.apache.org/jira/browse/FLINK-26904>LINK-26904
<https://issues.apache.org/jira/browse/FLINK-26904>.

Every Stage subclass should implement a static method with signature
"static T load(StreamTableEnvironment tEnv, String path)" ...

Thanks,
Dong


On Mon, Nov 15, 2021 at 4:09 PM Dong Lin <li...@gmail.com> wrote:

> Hi everyone,
>
> We would like to make the following API changes to this FLIP:
>
> 1) Updates Stage's Java doc to say the following
> Every Stage subclass should implement a static method with signature
> "static T load(StreamExecutionEnvironment env, String path)", where T
> refers to the concrete subclass. This static method should instantiate a
> new stage instance based on the data read from the given path.
>
> 2) Updates Model::setModelData(...) to return the Model instance itself.
>
> The reason for the first change is that the static load(...) method will
> need to create Table instances to read model data. And it needs to use the
> same StreamExecutionEnvironment to create those Table instances as the
> StreamExecutionEnvironment used in the rest of the program. The reason for
> the second change is to improve usability. This changes allows the users to
> do something like "return new KMeansModel.setModelData(...)"
>
> Please let us know if you have any suggestions on the proposed changes.
>
> Thanks!
> Dong
>
>
> On Mon, Sep 20, 2021 at 10:19 PM Dong Lin <li...@gmail.com> wrote:
>
>> Thanks everyone for your votes!
>>
>> The proposal has passed with the following votes:
>>
>> +1 (Binding): 3 (Stephan, Becket, Yun)
>> +1 (Non-binding): 0
>> -1: 0
>>
>> Thanks,
>> Dong
>>
>> On Sun, Sep 19, 2021 at 8:21 PM Yun Gao <yu...@aliyun.com.invalid>
>> wrote:
>>
>>> Very thanks Dong and Zhipeng for proposing the FLIP
>>> on Flink ML!
>>>
>>> The FLIP looks good to me and +1 from my side,
>>>
>>> Best,
>>> Yun
>>>
>>>
>>>
>>>  ------------------Original Mail ------------------
>>> Sender:Becket Qin <be...@gmail.com>
>>> Send Date:Sun Sep 19 12:35:39 2021
>>> Recipients:+dev <de...@flink.apache.org>
>>> Subject:Re: [VOTE] FLIP-173: Support DAG of algorithms
>>> +1 on the FLIP, really excited to see the relaunch of the Flink ML
>>> development!
>>>
>>> Thanks Dong and Zhipeng for driving this.
>>>
>>> Cheers,
>>>
>>> Jiangjie (Becket) Qin
>>>
>>> On Sat, Sep 18, 2021 at 12:17 AM Stephan Ewen <se...@apache.org> wrote:
>>>
>>> > I think this will be a useful addition.
>>> >
>>> > Regarding the API and specific design decisions: I think this looks ok.
>>> > I didn't dig very deep and would be fine to just go with the author's
>>> > proposal. The main motivation for having a separate flink-ml
>>> repository was
>>> > to develop more easily, make changes and iterate faster without having
>>> to
>>> > weight every design as carefully as we need to do it in core Flink.
>>> >
>>> > So +1 from my side
>>> >
>>> > On Fri, Sep 10, 2021 at 4:33 AM Dong Lin <li...@gmail.com> wrote:
>>> >
>>> > > Hi all,
>>> > >
>>> > > We would like to start the vote for FLIP-173: Support DAG of
>>> > > algorithms [1]. This FLIP was discussed in this thread [2].
>>> > >
>>> > > The proposal extended the Flink ML API to support DAG of algorithms
>>> where
>>> > > each algorithm could have multiple inputs and multiple outputs. It
>>> also
>>> > > extended Flink ML API to support online learning scenarios where a
>>> > > long-running Model instance needs to be continuously updated by the
>>> > latest
>>> > > model data generated by another long-running Estimator instance.
>>> > >
>>> > > The vote will be open for at least 72 hours, following the consensus
>>> > voting
>>> > > process.
>>> > >
>>> > > Thanks!
>>> > > Dong Lin and Zhipeng Zhang
>>> > >
>>> > > [1]
>>> > >
>>> >
>>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=184615783
>>> > > [2]
>>> > >
>>> > >
>>> >
>>> https://lists.apache.org/thread.html/r6729f351fb1bc13a93754c199d5fee1051cc8146e22374737c578779%40%3Cdev.flink.apache.org%3E
>>> > >
>>> >
>>>
>>