You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Huaxin Gao (Jira)" <ji...@apache.org> on 2020/02/25 18:41:00 UTC
[jira] [Comment Edited] (SPARK-30928) ML, GraphX 3.0 QA: API:
Binary incompatible changes
[ https://issues.apache.org/jira/browse/SPARK-30928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043874#comment-17043874 ]
Huaxin Gao edited comment on SPARK-30928 at 2/25/20 6:40 PM:
-------------------------------------------------------------
I audited all the ML, MLLIb, GraphX-related MiMa exclusions added for 3.0:
A few of them are not necessary. Will open a PR to remove those.
The following are false positive:
https://issues.apache.org/jira/browse/SPARK-16872 (private constructor)
https://issues.apache.org/jira/browse/SPARK-25838 (private or protected member variables)
https://issues.apache.org/jira/browse/SPARK-11215 (protected methods)
https://issues.apache.org/jira/browse/SPARK-26616 (private constructor or private object)
https://issues.apache.org/jira/browse/SPARK-25765 (private constructor)
The following are caused by inheritance structure change. The external APIs are still the same for users, so we don't need to document these in migration guide.
https://issues.apache.org/jira/browse/SPARK-29645 (the param is moved from individual algorithm to shared Params.
https://issues.apache.org/jira/browse/SPARK-28968 (the param is moved from individual algorithm to shared Params.
https://issues.apache.org/jira/browse/SPARK-30377 (AFT extends Estimator -> AFT extends Regressor)
Need to check migration guide for the following:
Remove deprecated APIs:
https://issues.apache.org/jira/browse/SPARK-28980
https://issues.apache.org/jira/browse/SPARK-27410
https://issues.apache.org/jira/browse/SPARK-26127
https://issues.apache.org/jira/browse/SPARK-26090
https://issues.apache.org/jira/browse/SPARK-25382
Binary incompatible changes
https://issues.apache.org/jira/browse/SPARK-28780
https://issues.apache.org/jira/browse/SPARK-26133
https://issues.apache.org/jira/browse/SPARK-30144
https://issues.apache.org/jira/browse/SPARK-30630
was (Author: huaxingao):
I went through the v30excludes list to check if everything on the list is required for exclusion. I found the following can be removed:
// ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.tree.HasVarianceImpurity.org$apache$spark$ml$tree$HasVarianceImpurity$_setter_$impurity_="),
// ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.tree.HasVarianceImpurity.org$apache$spark$ml$tree$HasVarianceImpurity$_setter_$impurity_="),
// ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.tree.HasVarianceImpurity.org$apache$spark$ml$tree$HasVarianceImpurity$_setter_$impurity_="),
// ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.tree.HasVarianceImpurity.org$apache$spark$ml$tree$HasVarianceImpurity$_setter_$impurity_="),
// ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("org.apache.spark.ml.tree.HasVarianceImpurity.org$apache$spark$ml$tree$HasVarianceImpurity$_setter_$impurity_="),
// ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.ml.feature.OneHotEncoder.getInputCol"),
// ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.ml.feature.OneHotEncoder.getOutputCol"),
// ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.ml.feature.OneHotEncoder.inputCol"),
// ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.ml.feature.OneHotEncoder.setInputCol"),
// ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.ml.feature.OneHotEncoder.setOutputCol"),
// ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.ml.feature.OneHotEncoder.outputCol"),
// ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.ml.image.ImageSchema.readImages"),
I will submit a PR to remove these. Is this sufficient for this ticket? Are there any other things I need to do? cc [~anders8]
> ML, GraphX 3.0 QA: API: Binary incompatible changes
> ---------------------------------------------------
>
> Key: SPARK-30928
> URL: https://issues.apache.org/jira/browse/SPARK-30928
> Project: Spark
> Issue Type: Sub-task
> Components: Documentation, GraphX, ML, MLlib
> Affects Versions: 3.0.0
> Reporter: zhengruifeng
> Priority: Blocker
>
> Generate a list of binary incompatible changes using MiMa and create new JIRAs for issues found. Filter out false positives as needed.
> If you want to take this task, look at the analogous task from the previous release QA, and ping the Assignee for advice.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org