[jira] [Commented] (FLINK-15847) Include flink-ml-api and flink-ml-lib in opt

["Hequn Cheng (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-15847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17028715#comment-17028715 ] 

Hequn Cheng commented on FLINK-15847:
-------------------------------------

We can add `maven-shade-plugin` in the pom of flink-ml-lib module to package the jar and then move the jar to the opt folder with the ability of flink-dist, i.e., adding a file config similar to sql-client in flink-dist/src/main/assemblies/opt.xml. The packaged jar would contain both classes of flink-ml-api and flink-ml-lib. 

> Include flink-ml-api and flink-ml-lib in opt
> --------------------------------------------
>
>                 Key: FLINK-15847
>                 URL: https://issues.apache.org/jira/browse/FLINK-15847
>             Project: Flink
>          Issue Type: Improvement
>          Components: Library / Machine Learning
>            Reporter: Hequn Cheng
>            Assignee: Hequn Cheng
>            Priority: Major
>
> [FLIP-39|https://cwiki.apache.org/confluence/display/FLINK/FLIP-39+Flink+ML+pipeline+and+ML+libs] rebuilds Flink ML pipeline on top of TableAPI which moves Flink ML a step further. Base on it, users can develop their ML jobs and more and more machine learning platforms are providing ML services.
> However, the problem now is the jars of flink-ml-api and flink-ml-lib are only exist on maven repo. Whenever users want to submit ML jobs, they can only depend on the ml modules and package a fat jar. This would be inconvenient especially for the machine learning platforms on which nearly all jobs depend on Flink ML modules and have to package a fat jar.
> Given this, it would be better to include jars of flink-ml-api and flink-ml-lib in the `opt` folder, so that users can directly use the jars with the binary release. For example, users can move the jars into the `lib` folder or use -j to upload the jars.
> Putting the jars in the `opt` folder instead of the `lib` folder is because currently, the ml jars are still optional for the Flink project by default.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)