[jira] [Created] (FLINK-11096) Create a new table based flink ML package

["Weihua Jiang (JIRA)" <ji...@apache.org>]

Weihua Jiang created FLINK-11096:
------------------------------------

             Summary: Create a new table based flink ML package
                 Key: FLINK-11096
                 URL: https://issues.apache.org/jira/browse/FLINK-11096
             Project: Flink
          Issue Type: Sub-task
          Components: Machine Learning Library, Table API & SQL
            Reporter: Weihua Jiang


Currently,  the DataSet based ML library is under org.apache._flink.ml_ scala package and under _flink-libraries/flink-ml directory._

 

There are two questions related to packaging:
 # Shall we create a new scala/java package, e.g. org.apache.flink.table.ml? Or still stay in org.apache.flink.ml?
 # Shall we still put new code in flink-libraries/flink-ml directory or create a new one, e.g. flink-libraries/flink-table-ml and corresponding maven package?

 

I implemented a prototype for the design and found that the new design is very hard to fit into existing flink.ml codebase. The existing flink.ml code is tightly coupled with DataSet API. Thus, I have to rewrite almost all parts of flink.ml to get some sample case to work. The only reusable code from flink.ml are the base math classes under _org.apache.flink.ml.math_ and _org.apache.flink.ml.metrics.distance_ packages. 

Considering this fact, I will prefer to create a new package org.apache.flink.table.ml and a new maven package flink-table-ml.

 

Please feel free to give your feedbacks. 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)