You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2023/01/14 09:05:48 UTC
[GitHub] [flink-ml] lindong28 opened a new pull request, #199: [demo-only] Refactor modules
lindong28 opened a new pull request, #199:
URL: https://github.com/apache/flink-ml/pull/199
This PR provides an example of refactoring flink-ml-core into two separate modules (i.e. flink-ml-core and flink-ml-servable-core), such that flink-ml-servable-core provides all the core functionalities needed to serve Flink ML model without depend on Flink runtime.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-ml] Fanoid commented on a diff in pull request #199: [demo-only] Refactor modules
Posted by "Fanoid (via GitHub)" <gi...@apache.org>.
Fanoid commented on code in PR #199:
URL: https://github.com/apache/flink-ml/pull/199#discussion_r1091397984
##########
flink-ml-servable-core/pom.xml:
##########
@@ -0,0 +1,74 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-ml-parent</artifactId>
+ <version>2.2-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>flink-ml-servable-core</artifactId>
+ <name>Flink ML : Servable : Core</name>
+
+ <dependencies>
+ <dependency>
Review Comment:
> We say moduleA is transitively depending on moduleC, if moduleA is directly depending on moduleB in its pom.xml, and moduleB is directly depending on moduleC in its pom.xml.
yeah. So in the two situations mentioned above, users are assumed to directly depend on `flink-ml-core-servable`, no matter depending on other modules (e.g. `flink-ml-core`) or not.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-ml] Fanoid commented on a diff in pull request #199: [demo-only] Refactor modules
Posted by "Fanoid (via GitHub)" <gi...@apache.org>.
Fanoid commented on code in PR #199:
URL: https://github.com/apache/flink-ml/pull/199#discussion_r1091386638
##########
flink-ml-servable-core/pom.xml:
##########
@@ -0,0 +1,74 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-ml-parent</artifactId>
+ <version>2.2-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>flink-ml-servable-core</artifactId>
+ <name>Flink ML : Servable : Core</name>
+
+ <dependencies>
+ <dependency>
Review Comment:
> > In both situations, after directly depending on flink-ml-servable-core, users may import DenseVectorTypeInfo in their code
>
> Hmm... can you explain why user would import DenseVectorTypeInfo in their code? I think users would only need to use DenseVector to use Flink ML, right? Also note that DenseVectorTypeInfo is not marked `@PublicEvolving`.
>
> BTW, I guess we need to mark classes such as `DenseVector` and `Vector` as `public`
For example, the prediction `Row` of `LogisticRegressionModel` contains a field of type `DenseVector` (storing probability of being positive and negative). It is possible that users call `flatMap(...).returns(Types.ROW(..., DenseVectorTypeInfo.INSTANCE, ...)` in their code to do some post-processing on the prediction results.
Searching `DenseVectorTypeInfo.INSTANCE` in Flink ML unit tests also gives some usages.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-ml] lindong28 commented on a diff in pull request #199: [demo-only] Refactor modules
Posted by "lindong28 (via GitHub)" <gi...@apache.org>.
lindong28 commented on code in PR #199:
URL: https://github.com/apache/flink-ml/pull/199#discussion_r1091363352
##########
flink-ml-servable-core/pom.xml:
##########
@@ -0,0 +1,74 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-ml-parent</artifactId>
+ <version>2.2-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>flink-ml-servable-core</artifactId>
+ <name>Flink ML : Servable : Core</name>
+
+ <dependencies>
+ <dependency>
Review Comment:
> In both situations, after directly depending on flink-ml-servable-core, users may import DenseVectorTypeInfo in their code
Hmm... can you explain why user would import DenseVectorTypeInfo in their code? I think users would only need to use DenseVector to use Flink ML, right? Also note that DenseVectorTypeInfo is not marked `@PublicEvolving`.
BTW, I guess we need to mark classes such as `DenseVector` and `Vector` as `public`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-ml] Fanoid commented on a diff in pull request #199: [demo-only] Refactor modules
Posted by "Fanoid (via GitHub)" <gi...@apache.org>.
Fanoid commented on code in PR #199:
URL: https://github.com/apache/flink-ml/pull/199#discussion_r1090122715
##########
flink-ml-servable-core/pom.xml:
##########
@@ -0,0 +1,74 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-ml-parent</artifactId>
+ <version>2.2-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>flink-ml-servable-core</artifactId>
+ <name>Flink ML : Servable : Core</name>
+
+ <dependencies>
+ <dependency>
Review Comment:
It seems not to meet the requirements of excluding all Flink libraries here, since we still take flink-core and flink-shaded-jackson as dependencies.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-ml] Fanoid commented on a diff in pull request #199: [demo-only] Refactor modules
Posted by "Fanoid (via GitHub)" <gi...@apache.org>.
Fanoid commented on code in PR #199:
URL: https://github.com/apache/flink-ml/pull/199#discussion_r1090567035
##########
flink-ml-servable-core/pom.xml:
##########
@@ -0,0 +1,74 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-ml-parent</artifactId>
+ <version>2.2-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>flink-ml-servable-core</artifactId>
+ <name>Flink ML : Servable : Core</name>
+
+ <dependencies>
+ <dependency>
Review Comment:
Hi, @lindong28 , I understand your point. For the issues mentioned above, I agree that we could figure out solutions sooner or later. My concern is the compatibility issue that will be introduced in the future when resolving above issues.
Assuming current PR is merged (after reviewed), people may start using `flink-ml-servable-core`. End-users are free to use `TypeInformation`s like `DenseVectorTypeInfo`, since they are public and necessary if they use `DenseVector` in Flink jobs. However, to achieve our goal, we have to eliminate `flink-core` from `flink-ml-servable-core` dependencies in future changes. The elimination is likely to lead to moving `DenseVectorTypeInfo` out from `flink-ml-servable-core`. As a consequence, this could breaks end-users' usages described above during this time.
Hope this can clarify my concern 😊.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-ml] lindong28 commented on a diff in pull request #199: [demo-only] Refactor modules
Posted by "lindong28 (via GitHub)" <gi...@apache.org>.
lindong28 commented on code in PR #199:
URL: https://github.com/apache/flink-ml/pull/199#discussion_r1091363352
##########
flink-ml-servable-core/pom.xml:
##########
@@ -0,0 +1,74 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-ml-parent</artifactId>
+ <version>2.2-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>flink-ml-servable-core</artifactId>
+ <name>Flink ML : Servable : Core</name>
+
+ <dependencies>
+ <dependency>
Review Comment:
> In both situations, after directly depending on flink-ml-servable-core, users may import DenseVectorTypeInfo in their code
Hmm... can you explain why user would import DenseVectorTypeInfo in their code? I think users would only need to use DenseVector to use Flink ML, right? Also note that DenseVectorTypeInfo is not marked `@public`.
BTW, I guess we need to mark classes such as `DenseVector` and `Vector` as `public`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-ml] Fanoid commented on a diff in pull request #199: [demo-only] Refactor modules
Posted by "Fanoid (via GitHub)" <gi...@apache.org>.
Fanoid commented on code in PR #199:
URL: https://github.com/apache/flink-ml/pull/199#discussion_r1091424389
##########
flink-ml-servable-core/pom.xml:
##########
@@ -0,0 +1,74 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-ml-parent</artifactId>
+ <version>2.2-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>flink-ml-servable-core</artifactId>
+ <name>Flink ML : Servable : Core</name>
+
+ <dependencies>
+ <dependency>
Review Comment:
> n the use-case mentioned above, users are using `LogisticRegressionModel`, which means they need to depend on `flink-ml-core` in their project's pom.xml. This means we can move `DenseVectorTypeInfo` from `flink-ml-servable-core` to `flink-ml-core` without breaking their code, right?
I see your point now: it's meaningless to use `DenseVectorTypeInfo` solely without depending on `flink-ml-core`. Actually, there is one rare situation that users call `DenseVectorTypeInfo#createSerializer` and use it just for serialization/deserialization of `DenseVector` with only depending on `flink-core`.
Nevertheless, I agree that future movement of `DenseVectorTypeInfo` will hardly break end-users' codes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-ml] lindong28 commented on a diff in pull request #199: [demo-only] Refactor modules
Posted by "lindong28 (via GitHub)" <gi...@apache.org>.
lindong28 commented on code in PR #199:
URL: https://github.com/apache/flink-ml/pull/199#discussion_r1090214298
##########
flink-ml-servable-core/pom.xml:
##########
@@ -0,0 +1,74 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-ml-parent</artifactId>
+ <version>2.2-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>flink-ml-servable-core</artifactId>
+ <name>Flink ML : Servable : Core</name>
+
+ <dependencies>
+ <dependency>
Review Comment:
@Fanoid I think `@PublicEvolving` tags whether an API is public or not. We can change this to a class defined in flink-ml-servable without having any impact on user (i.e. users program can still compile with Flink ML API/library without issue).
The @TypeInfo is only used inside the flink-ml-servable implementation. In the future Flink should provide API for users to register custom serializer/deserializer for selected classes. Then we can use this API in flink-ml-core and flink-ml-servable-core won't need to depend on `@TypeInfo`.
Thus for the two cases described above, we are free to update flink-ml-servable in a backward compatible way such that flink-ml-servable-core can avoid depending on flink runtime. What do you think?
##########
flink-ml-servable-core/pom.xml:
##########
@@ -0,0 +1,74 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-ml-parent</artifactId>
+ <version>2.2-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>flink-ml-servable-core</artifactId>
+ <name>Flink ML : Servable : Core</name>
+
+ <dependencies>
+ <dependency>
Review Comment:
@Fanoid I think `@PublicEvolving` tags whether an API is public or not. We can change this to a class defined in flink-ml-servable without having any impact on user (i.e. users program can still compile with Flink ML API/library without issue).
The `@TypeInfo` is only used inside the flink-ml-servable implementation. In the future Flink should provide API for users to register custom serializer/deserializer for selected classes. Then we can use this API in flink-ml-core and flink-ml-servable-core won't need to depend on `@TypeInfo`.
Thus for the two cases described above, we are free to update flink-ml-servable in a backward compatible way such that flink-ml-servable-core can avoid depending on flink runtime. What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-ml] Fanoid commented on a diff in pull request #199: [demo-only] Refactor modules
Posted by "Fanoid (via GitHub)" <gi...@apache.org>.
Fanoid commented on code in PR #199:
URL: https://github.com/apache/flink-ml/pull/199#discussion_r1091397984
##########
flink-ml-servable-core/pom.xml:
##########
@@ -0,0 +1,74 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-ml-parent</artifactId>
+ <version>2.2-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>flink-ml-servable-core</artifactId>
+ <name>Flink ML : Servable : Core</name>
+
+ <dependencies>
+ <dependency>
Review Comment:
> We say moduleA is transitively depending on moduleC, if moduleA is directly depending on moduleB in its pom.xml, and moduleB is directly depending on moduleC in its pom.xml.
yeah. So in the two situations mentioned above, users are assumed to directly depend on `flink-ml-core-servable`, no matter following the cited rule or not.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-ml] lindong28 commented on a diff in pull request #199: [demo-only] Refactor modules
Posted by "lindong28 (via GitHub)" <gi...@apache.org>.
lindong28 commented on code in PR #199:
URL: https://github.com/apache/flink-ml/pull/199#discussion_r1090214298
##########
flink-ml-servable-core/pom.xml:
##########
@@ -0,0 +1,74 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-ml-parent</artifactId>
+ <version>2.2-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>flink-ml-servable-core</artifactId>
+ <name>Flink ML : Servable : Core</name>
+
+ <dependencies>
+ <dependency>
Review Comment:
Hi @Fanoid, I think `@PublicEvolving` tags whether an API is public or not. We can change this to a class defined in flink-ml-servable without having any impact on user (i.e. users program can still compile with Flink ML API/library without issue).
The `@TypeInfo` is only used inside the flink-ml-servable implementation. In the future Flink should provide API for users to register custom serializer/deserializer for selected classes. Then we can use this API in flink-ml-core and flink-ml-servable-core won't need to depend on `@TypeInfo`.
Thus for the two cases described above, we are free to update flink-ml-servable in a backward compatible way such that flink-ml-servable-core can avoid depending on flink runtime. What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-ml] lindong28 commented on a diff in pull request #199: [demo-only] Refactor modules
Posted by "lindong28 (via GitHub)" <gi...@apache.org>.
lindong28 commented on code in PR #199:
URL: https://github.com/apache/flink-ml/pull/199#discussion_r1091402478
##########
flink-ml-servable-core/pom.xml:
##########
@@ -0,0 +1,74 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-ml-parent</artifactId>
+ <version>2.2-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>flink-ml-servable-core</artifactId>
+ <name>Flink ML : Servable : Core</name>
+
+ <dependencies>
+ <dependency>
Review Comment:
@Fanoid In the use-case mentioned above, users are using `LogisticRegressionModel`, which means they need to depend on `flink-ml-core` in their project's pom.xml. This means we can move `DenseVectorTypeInfo` from `flink-ml-servable-core` to `flink-ml-core` without breaking their code, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-ml] lindong28 commented on a diff in pull request #199: [demo-only] Refactor modules
Posted by "lindong28 (via GitHub)" <gi...@apache.org>.
lindong28 commented on code in PR #199:
URL: https://github.com/apache/flink-ml/pull/199#discussion_r1091476864
##########
flink-ml-servable-core/pom.xml:
##########
@@ -0,0 +1,74 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-ml-parent</artifactId>
+ <version>2.2-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>flink-ml-servable-core</artifactId>
+ <name>Flink ML : Servable : Core</name>
+
+ <dependencies>
+ <dependency>
Review Comment:
Cool. Thanks for all the comments!
BTW, with the current design of FLIP-289, users of TransformerServable will see types such as List, but not DenseVector. That means only users of Estimator/Transformer might need to call DenseVectorTypeInfo#createSerializer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-ml] Fanoid commented on a diff in pull request #199: [demo-only] Refactor modules
Posted by "Fanoid (via GitHub)" <gi...@apache.org>.
Fanoid commented on code in PR #199:
URL: https://github.com/apache/flink-ml/pull/199#discussion_r1091343636
##########
flink-ml-servable-core/pom.xml:
##########
@@ -0,0 +1,74 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-ml-parent</artifactId>
+ <version>2.2-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>flink-ml-servable-core</artifactId>
+ <name>Flink ML : Servable : Core</name>
+
+ <dependencies>
+ <dependency>
Review Comment:
Sorry I may missed something, as I can't figure out what the transitive dependency is in the case.
If following the rule you cited, users have to **directly** specify `flink-ml-servable-core` as a dependency to use classes like `DenseVector`, `DenseVectorTypeInfo`, and so on. If not following the rule, users still have the chance to **directly** depend on `flink-ml-servable-core` in their projects, because this module will be made public and be depended on without restrictions.
In both situations, after **directly** depending on `flink-ml-servable-core`, users may import `DenseVectorTypeInfo` in their code (because Flink requires type informations in some places). Note that there is no transitive dependency here. Then comes the breaks if we move `DenseVectorTypeInfo` out of `flink-ml-servable-core` in future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-ml] lindong28 commented on a diff in pull request #199: [demo-only] Refactor modules
Posted by "lindong28 (via GitHub)" <gi...@apache.org>.
lindong28 commented on code in PR #199:
URL: https://github.com/apache/flink-ml/pull/199#discussion_r1091402478
##########
flink-ml-servable-core/pom.xml:
##########
@@ -0,0 +1,74 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-ml-parent</artifactId>
+ <version>2.2-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>flink-ml-servable-core</artifactId>
+ <name>Flink ML : Servable : Core</name>
+
+ <dependencies>
+ <dependency>
Review Comment:
@Fanoid In the use-case mentioned above, users is using `LogisticRegressionModel`, which means they need to depend on `flink-ml-core` in their project's pom.xml. This means we can move `DenseVectorTypeInfo` from `flink-ml-servable-core` to `flink-ml-core` without breaking their code, right?
##########
flink-ml-servable-core/pom.xml:
##########
@@ -0,0 +1,74 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-ml-parent</artifactId>
+ <version>2.2-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>flink-ml-servable-core</artifactId>
+ <name>Flink ML : Servable : Core</name>
+
+ <dependencies>
+ <dependency>
Review Comment:
@Fanoid In the use-case mentioned above, the user is using `LogisticRegressionModel`, which means they need to depend on `flink-ml-core` in their project's pom.xml. This means we can move `DenseVectorTypeInfo` from `flink-ml-servable-core` to `flink-ml-core` without breaking their code, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-ml] Fanoid commented on a diff in pull request #199: [demo-only] Refactor modules
Posted by "Fanoid (via GitHub)" <gi...@apache.org>.
Fanoid commented on code in PR #199:
URL: https://github.com/apache/flink-ml/pull/199#discussion_r1090141977
##########
flink-ml-servable-core/pom.xml:
##########
@@ -0,0 +1,74 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-ml-parent</artifactId>
+ <version>2.2-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>flink-ml-servable-core</artifactId>
+ <name>Flink ML : Servable : Core</name>
+
+ <dependencies>
+ <dependency>
Review Comment:
Hi, thanks for your elaboration about my concern.
IMHO, I think our public APIs are still depending on flink-core somehow:
The annotation `@PublicEvolving` is imported from flink-core, which I think is part of the public APIs. The annotation `@TypeInfo` is also from flink-core. Without `@TypeInfo`, users have to manual register FlinkML types, like DenseVector, SparseVector, etc., and their `TypeInformation` s. (Maybe there are more such cases.)
So, maybe there is no need to make half-done changes public in a hurry, since they are likely to be modified significantly later (and affects users).
Actually, what I assumed is to resolve above issues in one PR, otherwise another PR is still required.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-ml] Fanoid commented on a diff in pull request #199: [demo-only] Refactor modules
Posted by "Fanoid (via GitHub)" <gi...@apache.org>.
Fanoid commented on code in PR #199:
URL: https://github.com/apache/flink-ml/pull/199#discussion_r1091386638
##########
flink-ml-servable-core/pom.xml:
##########
@@ -0,0 +1,74 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-ml-parent</artifactId>
+ <version>2.2-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>flink-ml-servable-core</artifactId>
+ <name>Flink ML : Servable : Core</name>
+
+ <dependencies>
+ <dependency>
Review Comment:
> > In both situations, after directly depending on flink-ml-servable-core, users may import DenseVectorTypeInfo in their code
>
> Hmm... can you explain why user would import DenseVectorTypeInfo in their code? I think users would only need to use DenseVector to use Flink ML, right? Also note that DenseVectorTypeInfo is not marked `@PublicEvolving`.
>
> BTW, I guess we need to mark classes such as `DenseVector` and `Vector` as `public`
For example, the prediction `Row` of `LogisticRegressionModel` contains a field of type `DenseVector` (storing probability of being positive and negative). It is possible that users call `flatMap(...).returns(Types.ROW(..., DenseVectorTypeInfo.INSTANCE, ...)` in their code to do some post-processing on the prediction results.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-ml] lindong28 commented on a diff in pull request #199: [demo-only] Refactor modules
Posted by "lindong28 (via GitHub)" <gi...@apache.org>.
lindong28 commented on code in PR #199:
URL: https://github.com/apache/flink-ml/pull/199#discussion_r1091314017
##########
flink-ml-servable-core/pom.xml:
##########
@@ -0,0 +1,74 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-ml-parent</artifactId>
+ <version>2.2-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>flink-ml-servable-core</artifactId>
+ <name>Flink ML : Servable : Core</name>
+
+ <dependencies>
+ <dependency>
Review Comment:
Thanks for the detailed explanation @Fanoid 😊
In the case you mentioned, user's code will be using a classA from a transitive Maven dependency without explicitly specifying this dependencies in pom.xml. This is a classic problem that has been discussed in various threads like this [1]. This issue is due to misbehavior from user side. The general solution as mentioned in [1] is that, "If your code depends on a module, your code should depends only directly on classes of this module, not a transitive-dependency of this module".
Note that it is not considered a public API change if we remove an internal dependency from a Flink's pom.xml. No FLIP is needed to make this change. I suppose we can also apply this rule for Flink ML?
[1] https://stackoverflow.com/questions/47260901/should-i-rely-on-transitive-dependencies-in-maven-if-they-come-from-other-sub-mo
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-ml] lindong28 commented on a diff in pull request #199: [demo-only] Refactor modules
Posted by "lindong28 (via GitHub)" <gi...@apache.org>.
lindong28 commented on code in PR #199:
URL: https://github.com/apache/flink-ml/pull/199#discussion_r1090214298
##########
flink-ml-servable-core/pom.xml:
##########
@@ -0,0 +1,74 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-ml-parent</artifactId>
+ <version>2.2-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>flink-ml-servable-core</artifactId>
+ <name>Flink ML : Servable : Core</name>
+
+ <dependencies>
+ <dependency>
Review Comment:
@Fanoid I think @PublicEvolving tags whether an API is public or not. We can change this to a class defined in flink-ml-servable without having any impact on user (i.e. users program can still compile with Flink ML API/library without issue).
The @TypeInfo is only used inside the flink-ml-servable implementation. In the future Flink should provide API for users to register custom serializer/deserializer for selected classes. Then we can use this API in flink-ml-core and flink-ml-servable-core won't need to depend on @TypeInfo.
Thus for the two cases described above, we are free to update flink-ml-servable in a backward compatible way such that flink-ml-servable-core can avoid depending on flink runtime. What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-ml] Fanoid commented on a diff in pull request #199: [demo-only] Refactor modules
Posted by "Fanoid (via GitHub)" <gi...@apache.org>.
Fanoid commented on code in PR #199:
URL: https://github.com/apache/flink-ml/pull/199#discussion_r1090567035
##########
flink-ml-servable-core/pom.xml:
##########
@@ -0,0 +1,74 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-ml-parent</artifactId>
+ <version>2.2-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>flink-ml-servable-core</artifactId>
+ <name>Flink ML : Servable : Core</name>
+
+ <dependencies>
+ <dependency>
Review Comment:
Hi, @lindong28 , I understand your point. For the issues mentioned above, I agree that we could figure out solutions sooner or later. My concern is the compatibility issue that will be introduced in the future when resolving above issues.
Assuming current PR is merged (after reviewed), people may start importing `flink-ml-servable-core` as a dependency of their project. End-users are free to use `TypeInformation`s like `DenseVectorTypeInfo`, since they are public and necessary if they use `DenseVector` in Flink jobs. However, to achieve our goal, we have to eliminate `flink-core` from `flink-ml-servable-core` dependencies in future changes. The elimination is likely to lead to moving `DenseVectorTypeInfo` out from `flink-ml-servable-core`. As a consequence, this could breaks end-users' usages described above during this time.
Hope this can clarify my concern 😊.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-ml] lindong28 commented on a diff in pull request #199: [demo-only] Refactor modules
Posted by "lindong28 (via GitHub)" <gi...@apache.org>.
lindong28 commented on code in PR #199:
URL: https://github.com/apache/flink-ml/pull/199#discussion_r1091364653
##########
flink-ml-servable-core/pom.xml:
##########
@@ -0,0 +1,74 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-ml-parent</artifactId>
+ <version>2.2-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>flink-ml-servable-core</artifactId>
+ <name>Flink ML : Servable : Core</name>
+
+ <dependencies>
+ <dependency>
Review Comment:
We say moduleA is transitively depending on moduleA, if moduleA is directly depending on moduleB in its pom.xml, and moduleB is directly depending on moduleC in its pom.xml.
##########
flink-ml-servable-core/pom.xml:
##########
@@ -0,0 +1,74 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-ml-parent</artifactId>
+ <version>2.2-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>flink-ml-servable-core</artifactId>
+ <name>Flink ML : Servable : Core</name>
+
+ <dependencies>
+ <dependency>
Review Comment:
We say moduleA is transitively depending on moduleC, if moduleA is directly depending on moduleB in its pom.xml, and moduleB is directly depending on moduleC in its pom.xml.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-ml] lindong28 commented on a diff in pull request #199: [demo-only] Refactor modules
Posted by "lindong28 (via GitHub)" <gi...@apache.org>.
lindong28 commented on code in PR #199:
URL: https://github.com/apache/flink-ml/pull/199#discussion_r1091314017
##########
flink-ml-servable-core/pom.xml:
##########
@@ -0,0 +1,74 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-ml-parent</artifactId>
+ <version>2.2-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>flink-ml-servable-core</artifactId>
+ <name>Flink ML : Servable : Core</name>
+
+ <dependencies>
+ <dependency>
Review Comment:
Thanks for the detailed explanation @Fanoid :)
In the case you mentioned, user's code will be using a classA from a transitive Maven dependency without explicitly specifying this dependencies in pom.xml. This is a classic problem that has been discussed in various threads like this [1]. This issue is due to misbehavior from user side. The general solution as mentioned in [1] is that, "If your code depends on a module, your code should depends only directly on classes of this module, not a transitive-dependency of this module".
Note that it is not considered a public API change if we remove an internal dependency from a Flink's pom.xml. No FLIP is needed to make this change. I suppose we can also apply this rule for Flink ML?
[1] https://stackoverflow.com/questions/47260901/should-i-rely-on-transitive-dependencies-in-maven-if-they-come-from-other-sub-mo
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-ml] lindong28 commented on a diff in pull request #199: [demo-only] Refactor modules
Posted by "lindong28 (via GitHub)" <gi...@apache.org>.
lindong28 commented on code in PR #199:
URL: https://github.com/apache/flink-ml/pull/199#discussion_r1090125842
##########
flink-ml-servable-core/pom.xml:
##########
@@ -0,0 +1,74 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-ml-parent</artifactId>
+ <version>2.2-SNAPSHOT</version>
+ </parent>
+
+ <artifactId>flink-ml-servable-core</artifactId>
+ <name>Flink ML : Servable : Core</name>
+
+ <dependencies>
+ <dependency>
Review Comment:
flinik-core is introduced here because we need to use the serializer/deserialize library in this module in the short term etc. for Flink ML 2.2 release. But since our public API does not depend on anything from flink-core or flink-shaded-jackson, we should be able to remove those dependencies without having any impact on users and thus achieve our goal in the long term.
Does this address your concern?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-ml] lindong28 closed pull request #199: [demo-only] Refactor modules
Posted by "lindong28 (via GitHub)" <gi...@apache.org>.
lindong28 closed pull request #199: [demo-only] Refactor modules
URL: https://github.com/apache/flink-ml/pull/199
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org