You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Nick Pentreath (JIRA)" <ji...@apache.org> on 2016/07/04 14:32:11 UTC
[jira] [Comment Edited] (SPARK-14810) ML, Graph 2.0 QA: API: Binary
incompatible changes
[ https://issues.apache.org/jira/browse/SPARK-14810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15293316#comment-15293316 ]
Nick Pentreath edited comment on SPARK-14810 at 7/4/16 2:31 PM:
----------------------------------------------------------------
List of changes since {{1.6.0}} audited - these are "false positives" due to being private, @Experimental, DeveloperAPI, etc:
* SPARK-13686 - Add a constructor parameter `regParam` to (Streaming)LinearRegressionWithSGD
* SPARK-13664 - Replace HadoopFsRelation with FileFormat
* SPARK-11622 - Make LibSVMRelation extends HadoopFsRelation and Add LibSVMOutputWriter
* SPARK-13920 - MIMA checks should apply to @Experimental and @DeveloperAPI APIs
* SPARK-11011 - UserDefinedType serialization should be strongly typed
* SPARK-13817 - Re-enable MiMA and removes object DataFrame
* SPARK-13927 - add row/column iterator to local matrices - (add methods to sealed trait)
* SPARK-13948 - MiMa Check should catch if the visibility change to `private` - (DataFrame -> Dataset)
* SPARK-11262 - Unit test for gradient, loss layers, memory management - (private class)
* SPARK-13430 - moved featureCol from LinearRegressionModelSummary to LinearRegressionSummary - (private class)
* SPARK-13048 - keepLastCheckpoint option for LDA EM optimizer - (private class)
* SPARK-14734 - Add conversions between mllib and ml Vector, Matrix types - (private methods added)
* SPARK-14861 - Replace internal usages of SQLContext with SparkSession - (private class)
Binary incompatible changes:
* SPARK-14615 - Use new ML Vector and Matrix in pipeline API
** params:
*** {{scalingVec}} in {{ElementwiseProduct}}
** models:
*** {{coefficients}} in {{LinearRegressionModel}}, {{LogisticRegressionModel}} and {{AFTSurvivalRegressionModel}}
*** {{predict}}/{{predictQuantiles}} in {{AFTSurvivalRegressionModel}}
*** {{boundaries}}/{{predictions}} in {{IsotonicRegressionModel}}
*** {{pc}} in {{PCAModel}}
*** {{idf}} in {{IDFModel}}
*** {{originalMin}}/{{originalMax}} in {{MinMaxScalerModel}}
*** {{mean}}/{{std}} in {{StandardScalerModel}}
*** {{findSynonyms}} in {{Word2VecModel}}
*** {{pi}}/{{theta}} in {{NaiveBayesModel}}
*** {{weights}} in {{MultilayerPerceptronClassificationModel}}
*** {{clusterCenters}} in {{KMeansModel}}
*** {{estimatedDocConcentration}}/{{topicsMatrix}} in {{LDAModel}}
* SPARK-14814 - Fix the java compatibility issue for the output of {{spark.mllib.tree.model.DecisionTreeModel.predict}} method.
* SPARK-14089 - Remove methods that has been deprecated since 1.1, 1.2, 1.3, 1.4, and 1.5
* SPARK-14952 - Remove methods deprecated in 1.6
* {{DataFrame}} -> {{Dataset<Row>}} changes (this of course applies for all of Spark SQL)
was (Author: mlnick):
List of changes since {{1.6.0}} audited - these are "false positives" due to being private, @Experimental, DeveloperAPI, etc:
* SPARK-13686 - Add a constructor parameter `regParam` to (Streaming)LinearRegressionWithSGD
* SPARK-13664 - Replace HadoopFsRelation with FileFormat
* SPARK-11622 - Make LibSVMRelation extends HadoopFsRelation and Add LibSVMOutputWriter
* SPARK-13920 - MIMA checks should apply to @Experimental and @DeveloperAPI APIs
* SPARK-11011 - UserDefinedType serialization should be strongly typed
* SPARK-13817 - Re-enable MiMA and removes object DataFrame
* SPARK-13927 - add row/column iterator to local matrices - (add methods to sealed trait)
* SPARK-13948 - MiMa Check should catch if the visibility change to `private` - (DataFrame -> Dataset)
* SPARK-11262 - Unit test for gradient, loss layers, memory management - (private class)
* SPARK-13430 - moved featureCol from LinearRegressionModelSummary to LinearRegressionSummary - (private class)
* SPARK-13048 - keepLastCheckpoint option for LDA EM optimizer - (private class)
* SPARK-14734 - Add conversions between mllib and ml Vector, Matrix types - (private methods added)
* SPARK-14861 - Replace internal usages of SQLContext with SparkSession - (private class)
Binary incompatible changes:
* SPARK-14615 - Use new ML Vector and Matrix in pipeline API
** params:
*** {{scalingVec}} in {{ElementwiseProduct}}
** models:
*** {{coefficients}} in {{LinearRegressionModel}}, {{LogisticRegressionModel}} and {{AFTSurvivalRegressionModel}}
*** {{predict}}/{{predictQuantiles}} in {{AFTSurvivalRegressionModel}}
*** {{boundaries}}/{{predictions}} in {{IsotonicRegressionModel}}
*** {{pc}} in {{PCAModel}}
*** {{idf}} in {{IDFModel}}
*** {{originalMin}}/{{originalMax}} in {{MinMaxScalerModel}}
*** {{mean}}/{{std}} in {{StandardScalerModel}}
*** {{findSynonyms}} in {{Word2VecModel}}
*** {{pi}}/{{theta}} in {{NaiveBayesModel}}
*** {{weights}} in {{MultilayerPerceptronClassificationModel}}
*** {{clusterCenters}} in {{KMeansModel}}
*** {{estimatedDocConcentration}}/{{topicsMatrix}} in {{LDAModel}}
* SPARK-14814 - Fix the java compatibility issue for the output of {{spark.mllib.tree.model.DecisionTreeModel.predict}} method.
* SPARK-14089 - Remove methods that has been deprecated since 1.1, 1.2, 1.3, 1.4, and 1.5
* SPARK-14952 - Remove methods deprecated in 1.6
* DataFrame -> Dataset<Row> changes for Java (this of course applies for all of Spark SQL)
> ML, Graph 2.0 QA: API: Binary incompatible changes
> --------------------------------------------------
>
> Key: SPARK-14810
> URL: https://issues.apache.org/jira/browse/SPARK-14810
> Project: Spark
> Issue Type: Sub-task
> Components: Documentation, GraphX, ML, MLlib
> Reporter: Joseph K. Bradley
> Assignee: Nick Pentreath
> Priority: Blocker
>
> Generate a list of binary incompatible changes using MiMa and create new JIRAs for issues found. Filter out false positives as needed.
> If you want to take this task, look at the analogous task from the previous release QA, and ping the Assignee for advice.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org