You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Bonino Dario <da...@gmail.com> on 2022/01/17 12:39:22 UTC
Examples / Documentation for Flink ML 2
Dear List,
We are in the process of evaluating Flink ML version 2.0 in the context
of some ML task mainly concerned with classification and clustering.
While algorithms for this 2 domains are already present, although in a
limited form (perhaps) in the latest release of Flink ML, we did not
found any example / documentation that could guide our experiments.
Is some adoption example available, like code, tutorial or any
information that might help us in bootstrapping a Flink ML 2 project?
Thank you very much
Best regards
--
Ing. Dario Bonino, Ph.D
e-m@il:dario.bonino@gmail.com
www:https://www.linkedin.com/in/dariobonino
<foaf:Person>
<foaf:firstName>Dario</foaf:firstName>
<foaf:surname>Bonino</foaf:surname>
<foaf:msnChatID>slide473@hotmail.com</foaf:msnChatID>
</foaf:Person>
Re: Examples / Documentation for Flink ML 2
Posted by Dong Lin <li...@gmail.com>.
Hey Bonino,
Sounds great. Since we have not set up the website for Flink ML yet, how
about we create PRs for https://github.com/apache/flink-ml and put those
Markdown files under flink-ml/docs?
Best Regards,
Dong
On Sat, Jan 22, 2022 at 12:25 AM Bonino Dario <da...@gmail.com>
wrote:
> Hi Dong,
>
> We assembled a first, very small, Markdown document providing a jump-start
> description using a kMeans example. I could already share it with you to
> check if we are pointing in the right direction. I had a look at the Flink
> contribution guidelines, however the flink-ml project is somewhat
> "separate" from Flink and the same I think holds for the documentation. How
> do you think it is better to proceed?
>
> Best regards
>
> Dario Bonino
> On 1/19/22 09:36, Dong Lin wrote:
>
> Hi Bonino,
>
> Definitely, it will be great to build up the Flink ML docs together based
> on your experience.
>
> Thanks!
> Dong
>
> On Wed, Jan 19, 2022 at 4:32 PM Bonino Dario <da...@gmail.com>
> wrote:
>
>> Hi Dong,
>>
>> Thank you for the reply. Since we are actually experimenting with the
>> Flink ML libraries, If you think it's worth, we may contribute some
>> documentation, e.g., tutorial based on what we learn while setting up our
>> test project with Flink ML. Is it something that might be of interest for
>> you?
>>
>> Best regards
>>
>> Dario
>> On 1/18/22 04:51, Dong Lin wrote:
>>
>> Hi Bonino,
>>
>> Thanks for your interest!
>>
>> Flink ML is currently ready for experienced algorithm developers to try
>> it out because we have setup the basic APIs and infrastructure to develop
>> algorithms. Five algorithms (i.e. kmeans, naive bays, knn, logistic
>> regression and one-hot encoder) has been implemented in the last release.
>> Their unit tests can be found here
>> <https://github.com/apache/flink-ml/tree/master/flink-ml-lib/src/test/java/org/apache/flink/ml/feature>,
>> here
>> <https://github.com/apache/flink-ml/tree/master/flink-ml-lib/src/test/java/org/apache/flink/ml/clustering>
>> and here
>> <https://github.com/apache/flink-ml/tree/master/flink-ml-lib/src/test/java/org/apache/flink/ml/classification>,
>> which show how to use these algorithms (including transform/fit/save/load).
>> And from these unit tests you can find implementation of these algorithms
>> which can be used as reference implementation to develop other algorithms
>> of your interest.
>>
>> We plan to setup a website for Flink ML to provide links to
>> example/tutorial similar to the Flink Statefun website (link
>> <https://nightlies.apache.org/flink/flink-statefun-docs-stable/>). This
>> website will likely be setup in March. We are currently working on
>> developing further infrastructure for benchmarking and optimizing the
>> machine learning algorithms in Flink ML.
>>
>> Best Regards,
>> Dong
>>
>>
>>
>> On Mon, Jan 17, 2022 at 8:57 PM Dawid Wysakowicz <dw...@apache.org>
>> wrote:
>>
>>> I am adding a couple of people who worked on it. Hopefully, they will be
>>> able to answer you.
>>> On 17/01/2022 13:39, Bonino Dario wrote:
>>>
>>> Dear List,
>>>
>>> We are in the process of evaluating Flink ML version 2.0 in the context
>>> of some ML task mainly concerned with classification and clustering.
>>>
>>> While algorithms for this 2 domains are already present, although in a
>>> limited form (perhaps) in the latest release of Flink ML, we did not found
>>> any example / documentation that could guide our experiments.
>>>
>>> Is some adoption example available, like code, tutorial or any
>>> information that might help us in bootstrapping a Flink ML 2 project?
>>>
>>> Thank you very much
>>>
>>> Best regards
>>>
>>> --
>>> Ing. Dario Bonino, Ph.D
>>>
>>> e-m@il: dario.bonino@gmail.com
>>> www: https://www.linkedin.com/in/dariobonino
>>> <foaf:Person>
>>> <foaf:firstName>Dario</foaf:firstName>
>>> <foaf:surname>Bonino</foaf:surname>
>>> <foaf:msnChatID>slide473@hotmail.com</foaf:msnChatID>
>>> </foaf:Person>
>>>
>>> --
>> Ing. Dario Bonino, Ph.D
>>
>> e-m@il: dario.bonino@gmail.com
>> www: https://www.linkedin.com/in/dariobonino
>> <foaf:Person>
>> <foaf:firstName>Dario</foaf:firstName>
>> <foaf:surname>Bonino</foaf:surname>
>> <foaf:msnChatID>slide473@hotmail.com</foaf:msnChatID>
>> </foaf:Person>
>>
>> --
> Ing. Dario Bonino, Ph.D
>
> e-m@il: dario.bonino@gmail.com
> www: https://www.linkedin.com/in/dariobonino
> <foaf:Person>
> <foaf:firstName>Dario</foaf:firstName>
> <foaf:surname>Bonino</foaf:surname>
> <foaf:msnChatID>slide473@hotmail.com</foaf:msnChatID>
> </foaf:Person>
>
>
Re: Examples / Documentation for Flink ML 2
Posted by Bonino Dario <da...@gmail.com>.
Hi Dong,
We assembled a first, very small, Markdown document providing a
jump-start description using a kMeans example. I could already share it
with you to check if we are pointing in the right direction. I had a
look at the Flink contribution guidelines, however the flink-ml project
is somewhat "separate" from Flink and the same I think holds for the
documentation. How do you think it is better to proceed?
Best regards
Dario Bonino
On 1/19/22 09:36, Dong Lin wrote:
> Hi Bonino,
>
> Definitely, it will be great to build up the Flink ML docs together
> based on your experience.
>
> Thanks!
> Dong
>
> On Wed, Jan 19, 2022 at 4:32 PM Bonino Dario <da...@gmail.com>
> wrote:
>
> Hi Dong,
>
> Thank you for the reply. Since we are actually experimenting with
> the Flink ML libraries, If you think it's worth, we may contribute
> some documentation, e.g., tutorial based on what we learn while
> setting up our test project with Flink ML. Is it something that
> might be of interest for you?
>
> Best regards
>
> Dario
>
> On 1/18/22 04:51, Dong Lin wrote:
>> Hi Bonino,
>>
>> Thanks for your interest!
>>
>> Flink ML is currently ready for experienced algorithm developers
>> to try it out because we have setup the basic APIs and
>> infrastructure to develop algorithms. Five algorithms (i.e.
>> kmeans, naive bays, knn, logistic regression and one-hot encoder)
>> has been implemented in the last release. Their unit tests can be
>> found here
>> <https://github.com/apache/flink-ml/tree/master/flink-ml-lib/src/test/java/org/apache/flink/ml/feature>,
>> here
>> <https://github.com/apache/flink-ml/tree/master/flink-ml-lib/src/test/java/org/apache/flink/ml/clustering>
>> and here
>> <https://github.com/apache/flink-ml/tree/master/flink-ml-lib/src/test/java/org/apache/flink/ml/classification>,
>> which show how to use these algorithms (including
>> transform/fit/save/load). And from these unit tests you can find
>> implementation of these algorithms which can be used as reference
>> implementation to develop other algorithms of your interest.
>>
>> We plan to setup a website for Flink ML to provide links to
>> example/tutorial similar to the Flink Statefun website (link
>> <https://nightlies.apache.org/flink/flink-statefun-docs-stable/>).
>> This website will likely be setup in March. We are currently
>> working on developing further infrastructure for benchmarking and
>> optimizing the machine learning algorithms in Flink ML.
>>
>> Best Regards,
>> Dong
>>
>>
>>
>> On Mon, Jan 17, 2022 at 8:57 PM Dawid Wysakowicz
>> <dw...@apache.org> wrote:
>>
>> I am adding a couple of people who worked on it. Hopefully,
>> they will be able to answer you.
>>
>> On 17/01/2022 13:39, Bonino Dario wrote:
>>>
>>> Dear List,
>>>
>>> We are in the process of evaluating Flink ML version 2.0 in
>>> the context of some ML task mainly concerned with
>>> classification and clustering.
>>>
>>> While algorithms for this 2 domains are already present,
>>> although in a limited form (perhaps) in the latest release
>>> of Flink ML, we did not found any example / documentation
>>> that could guide our experiments.
>>>
>>> Is some adoption example available, like code, tutorial or
>>> any information that might help us in bootstrapping a Flink
>>> ML 2 project?
>>>
>>> Thank you very much
>>>
>>> Best regards
>>>
>>> --
>>> Ing. Dario Bonino, Ph.D
>>>
>>> e-m@il:dario.bonino@gmail.com
>>> www:https://www.linkedin.com/in/dariobonino
>>> <foaf:Person>
>>> <foaf:firstName>Dario</foaf:firstName>
>>> <foaf:surname>Bonino</foaf:surname>
>>> <foaf:msnChatID>slide473@hotmail.com</foaf:msnChatID>
>>> </foaf:Person>
>>
> --
> Ing. Dario Bonino, Ph.D
>
> e-m@il:dario.bonino@gmail.com
> www:https://www.linkedin.com/in/dariobonino
> <foaf:Person>
> <foaf:firstName>Dario</foaf:firstName>
> <foaf:surname>Bonino</foaf:surname>
> <foaf:msnChatID>slide473@hotmail.com</foaf:msnChatID>
> </foaf:Person>
>
--
Ing. Dario Bonino, Ph.D
e-m@il:dario.bonino@gmail.com
www:https://www.linkedin.com/in/dariobonino
<foaf:Person>
<foaf:firstName>Dario</foaf:firstName>
<foaf:surname>Bonino</foaf:surname>
<foaf:msnChatID>slide473@hotmail.com</foaf:msnChatID>
</foaf:Person>
Re: Examples / Documentation for Flink ML 2
Posted by Dong Lin <li...@gmail.com>.
Hi Bonino,
Definitely, it will be great to build up the Flink ML docs together based
on your experience.
Thanks!
Dong
On Wed, Jan 19, 2022 at 4:32 PM Bonino Dario <da...@gmail.com> wrote:
> Hi Dong,
>
> Thank you for the reply. Since we are actually experimenting with the
> Flink ML libraries, If you think it's worth, we may contribute some
> documentation, e.g., tutorial based on what we learn while setting up our
> test project with Flink ML. Is it something that might be of interest for
> you?
>
> Best regards
>
> Dario
> On 1/18/22 04:51, Dong Lin wrote:
>
> Hi Bonino,
>
> Thanks for your interest!
>
> Flink ML is currently ready for experienced algorithm developers to try it
> out because we have setup the basic APIs and infrastructure to develop
> algorithms. Five algorithms (i.e. kmeans, naive bays, knn, logistic
> regression and one-hot encoder) has been implemented in the last release.
> Their unit tests can be found here
> <https://github.com/apache/flink-ml/tree/master/flink-ml-lib/src/test/java/org/apache/flink/ml/feature>,
> here
> <https://github.com/apache/flink-ml/tree/master/flink-ml-lib/src/test/java/org/apache/flink/ml/clustering>
> and here
> <https://github.com/apache/flink-ml/tree/master/flink-ml-lib/src/test/java/org/apache/flink/ml/classification>,
> which show how to use these algorithms (including transform/fit/save/load).
> And from these unit tests you can find implementation of these algorithms
> which can be used as reference implementation to develop other algorithms
> of your interest.
>
> We plan to setup a website for Flink ML to provide links to
> example/tutorial similar to the Flink Statefun website (link
> <https://nightlies.apache.org/flink/flink-statefun-docs-stable/>). This
> website will likely be setup in March. We are currently working on
> developing further infrastructure for benchmarking and optimizing the
> machine learning algorithms in Flink ML.
>
> Best Regards,
> Dong
>
>
>
> On Mon, Jan 17, 2022 at 8:57 PM Dawid Wysakowicz <dw...@apache.org>
> wrote:
>
>> I am adding a couple of people who worked on it. Hopefully, they will be
>> able to answer you.
>> On 17/01/2022 13:39, Bonino Dario wrote:
>>
>> Dear List,
>>
>> We are in the process of evaluating Flink ML version 2.0 in the context
>> of some ML task mainly concerned with classification and clustering.
>>
>> While algorithms for this 2 domains are already present, although in a
>> limited form (perhaps) in the latest release of Flink ML, we did not found
>> any example / documentation that could guide our experiments.
>>
>> Is some adoption example available, like code, tutorial or any
>> information that might help us in bootstrapping a Flink ML 2 project?
>>
>> Thank you very much
>>
>> Best regards
>>
>> --
>> Ing. Dario Bonino, Ph.D
>>
>> e-m@il: dario.bonino@gmail.com
>> www: https://www.linkedin.com/in/dariobonino
>> <foaf:Person>
>> <foaf:firstName>Dario</foaf:firstName>
>> <foaf:surname>Bonino</foaf:surname>
>> <foaf:msnChatID>slide473@hotmail.com</foaf:msnChatID>
>> </foaf:Person>
>>
>> --
> Ing. Dario Bonino, Ph.D
>
> e-m@il: dario.bonino@gmail.com
> www: https://www.linkedin.com/in/dariobonino
> <foaf:Person>
> <foaf:firstName>Dario</foaf:firstName>
> <foaf:surname>Bonino</foaf:surname>
> <foaf:msnChatID>slide473@hotmail.com</foaf:msnChatID>
> </foaf:Person>
>
>
Re: Examples / Documentation for Flink ML 2
Posted by Bonino Dario <da...@gmail.com>.
Hi Dong,
Thank you for the reply. Since we are actually experimenting with the
Flink ML libraries, If you think it's worth, we may contribute some
documentation, e.g., tutorial based on what we learn while setting up
our test project with Flink ML. Is it something that might be of
interest for you?
Best regards
Dario
On 1/18/22 04:51, Dong Lin wrote:
> Hi Bonino,
>
> Thanks for your interest!
>
> Flink ML is currently ready for experienced algorithm developers to
> try it out because we have setup the basic APIs and infrastructure to
> develop algorithms. Five algorithms (i.e. kmeans, naive bays, knn,
> logistic regression and one-hot encoder) has been implemented in the
> last release. Their unit tests can be found here
> <https://github.com/apache/flink-ml/tree/master/flink-ml-lib/src/test/java/org/apache/flink/ml/feature>,
> here
> <https://github.com/apache/flink-ml/tree/master/flink-ml-lib/src/test/java/org/apache/flink/ml/clustering>
> and here
> <https://github.com/apache/flink-ml/tree/master/flink-ml-lib/src/test/java/org/apache/flink/ml/classification>,
> which show how to use these algorithms (including
> transform/fit/save/load). And from these unit tests you can find
> implementation of these algorithms which can be used as reference
> implementation to develop other algorithms of your interest.
>
> We plan to setup a website for Flink ML to provide links to
> example/tutorial similar to the Flink Statefun website (link
> <https://nightlies.apache.org/flink/flink-statefun-docs-stable/>).
> This website will likely be setup in March. We are currently working
> on developing further infrastructure for benchmarking and optimizing
> the machine learning algorithms in Flink ML.
>
> Best Regards,
> Dong
>
>
>
> On Mon, Jan 17, 2022 at 8:57 PM Dawid Wysakowicz
> <dw...@apache.org> wrote:
>
> I am adding a couple of people who worked on it. Hopefully, they
> will be able to answer you.
>
> On 17/01/2022 13:39, Bonino Dario wrote:
>>
>> Dear List,
>>
>> We are in the process of evaluating Flink ML version 2.0 in the
>> context of some ML task mainly concerned with classification and
>> clustering.
>>
>> While algorithms for this 2 domains are already present, although
>> in a limited form (perhaps) in the latest release of Flink ML, we
>> did not found any example / documentation that could guide our
>> experiments.
>>
>> Is some adoption example available, like code, tutorial or any
>> information that might help us in bootstrapping a Flink ML 2
>> project?
>>
>> Thank you very much
>>
>> Best regards
>>
>> --
>> Ing. Dario Bonino, Ph.D
>>
>> e-m@il:dario.bonino@gmail.com
>> www:https://www.linkedin.com/in/dariobonino
>> <foaf:Person>
>> <foaf:firstName>Dario</foaf:firstName>
>> <foaf:surname>Bonino</foaf:surname>
>> <foaf:msnChatID>slide473@hotmail.com</foaf:msnChatID>
>> </foaf:Person>
>
--
Ing. Dario Bonino, Ph.D
e-m@il:dario.bonino@gmail.com
www:https://www.linkedin.com/in/dariobonino
<foaf:Person>
<foaf:firstName>Dario</foaf:firstName>
<foaf:surname>Bonino</foaf:surname>
<foaf:msnChatID>slide473@hotmail.com</foaf:msnChatID>
</foaf:Person>
Re: Examples / Documentation for Flink ML 2
Posted by Dong Lin <li...@gmail.com>.
Hi Bonino,
Thanks for your interest!
Flink ML is currently ready for experienced algorithm developers to try it
out because we have setup the basic APIs and infrastructure to develop
algorithms. Five algorithms (i.e. kmeans, naive bays, knn, logistic
regression and one-hot encoder) has been implemented in the last release.
Their unit tests can be found here
<https://github.com/apache/flink-ml/tree/master/flink-ml-lib/src/test/java/org/apache/flink/ml/feature>,
here
<https://github.com/apache/flink-ml/tree/master/flink-ml-lib/src/test/java/org/apache/flink/ml/clustering>
and here
<https://github.com/apache/flink-ml/tree/master/flink-ml-lib/src/test/java/org/apache/flink/ml/classification>,
which show how to use these algorithms (including transform/fit/save/load).
And from these unit tests you can find implementation of these algorithms
which can be used as reference implementation to develop other algorithms
of your interest.
We plan to setup a website for Flink ML to provide links to
example/tutorial similar to the Flink Statefun website (link
<https://nightlies.apache.org/flink/flink-statefun-docs-stable/>). This
website will likely be setup in March. We are currently working on
developing further infrastructure for benchmarking and optimizing the
machine learning algorithms in Flink ML.
Best Regards,
Dong
On Mon, Jan 17, 2022 at 8:57 PM Dawid Wysakowicz <dw...@apache.org>
wrote:
> I am adding a couple of people who worked on it. Hopefully, they will be
> able to answer you.
> On 17/01/2022 13:39, Bonino Dario wrote:
>
> Dear List,
>
> We are in the process of evaluating Flink ML version 2.0 in the context of
> some ML task mainly concerned with classification and clustering.
>
> While algorithms for this 2 domains are already present, although in a
> limited form (perhaps) in the latest release of Flink ML, we did not found
> any example / documentation that could guide our experiments.
>
> Is some adoption example available, like code, tutorial or any information
> that might help us in bootstrapping a Flink ML 2 project?
>
> Thank you very much
>
> Best regards
>
> --
> Ing. Dario Bonino, Ph.D
>
> e-m@il: dario.bonino@gmail.com
> www: https://www.linkedin.com/in/dariobonino
> <foaf:Person>
> <foaf:firstName>Dario</foaf:firstName>
> <foaf:surname>Bonino</foaf:surname>
> <foaf:msnChatID>slide473@hotmail.com</foaf:msnChatID>
> </foaf:Person>
>
>
Re: Examples / Documentation for Flink ML 2
Posted by Dawid Wysakowicz <dw...@apache.org>.
I am adding a couple of people who worked on it. Hopefully, they will be
able to answer you.
On 17/01/2022 13:39, Bonino Dario wrote:
>
> Dear List,
>
> We are in the process of evaluating Flink ML version 2.0 in the
> context of some ML task mainly concerned with classification and
> clustering.
>
> While algorithms for this 2 domains are already present, although in a
> limited form (perhaps) in the latest release of Flink ML, we did not
> found any example / documentation that could guide our experiments.
>
> Is some adoption example available, like code, tutorial or any
> information that might help us in bootstrapping a Flink ML 2 project?
>
> Thank you very much
>
> Best regards
>
> --
> Ing. Dario Bonino, Ph.D
>
> e-m@il: dario.bonino@gmail.com
> www: https://www.linkedin.com/in/dariobonino
> <foaf:Person>
> <foaf:firstName>Dario</foaf:firstName>
> <foaf:surname>Bonino</foaf:surname>
> <foaf:msnChatID>slide473@hotmail.com</foaf:msnChatID>
> </foaf:Person>