You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by jk...@apache.org on 2016/03/08 22:27:39 UTC
spark git commit: [ML] testEstimatorAndModelReadWrite should call
checkModelData
Repository: spark
Updated Branches:
refs/heads/master 46881b4ea -> 9740954f3
[ML] testEstimatorAndModelReadWrite should call checkModelData
## What changes were proposed in this pull request?
Although we defined ```checkModelData``` in [```read/write``` test](https://github.com/apache/spark/blob/master/mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala#L994) of ML estimators/models and pass it to ```testEstimatorAndModelReadWrite```, ```testEstimatorAndModelReadWrite``` omits to call ```checkModelData``` to check the equality of model data. So actually we did not run the check of model data equality for all test cases currently, we should fix it.
BTW, fix the bug of LDA read/write test which did not set ```docConcentration```. This bug should have failed test, but it does not complain because we did not run ```checkModelData``` actually.
cc jkbradley mengxr
## How was this patch tested?
No new unit test, should pass the exist ones.
Author: Yanbo Liang <yb...@gmail.com>
Closes #11513 from yanboliang/ml-check-model-data.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/9740954f
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/9740954f
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/9740954f
Branch: refs/heads/master
Commit: 9740954f3feb9878cd263feca4aae628e1457f8b
Parents: 46881b4
Author: Yanbo Liang <yb...@gmail.com>
Authored: Tue Mar 8 13:27:31 2016 -0800
Committer: Joseph K. Bradley <jo...@databricks.com>
Committed: Tue Mar 8 13:27:31 2016 -0800
----------------------------------------------------------------------
.../src/test/scala/org/apache/spark/ml/clustering/LDASuite.scala | 3 ++-
.../scala/org/apache/spark/ml/util/DefaultReadWriteTest.scala | 3 +++
2 files changed, 5 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/9740954f/mllib/src/test/scala/org/apache/spark/ml/clustering/LDASuite.scala
----------------------------------------------------------------------
diff --git a/mllib/src/test/scala/org/apache/spark/ml/clustering/LDASuite.scala b/mllib/src/test/scala/org/apache/spark/ml/clustering/LDASuite.scala
index 0327040..a3a8f65 100644
--- a/mllib/src/test/scala/org/apache/spark/ml/clustering/LDASuite.scala
+++ b/mllib/src/test/scala/org/apache/spark/ml/clustering/LDASuite.scala
@@ -52,7 +52,8 @@ object LDASuite {
"checkpointInterval" -> 30,
"learningOffset" -> 1023.0,
"learningDecay" -> 0.52,
- "subsamplingRate" -> 0.051
+ "subsamplingRate" -> 0.051,
+ "docConcentration" -> Array(2.0)
)
}
http://git-wip-us.apache.org/repos/asf/spark/blob/9740954f/mllib/src/test/scala/org/apache/spark/ml/util/DefaultReadWriteTest.scala
----------------------------------------------------------------------
diff --git a/mllib/src/test/scala/org/apache/spark/ml/util/DefaultReadWriteTest.scala b/mllib/src/test/scala/org/apache/spark/ml/util/DefaultReadWriteTest.scala
index 0aa774b..8e5365a 100644
--- a/mllib/src/test/scala/org/apache/spark/ml/util/DefaultReadWriteTest.scala
+++ b/mllib/src/test/scala/org/apache/spark/ml/util/DefaultReadWriteTest.scala
@@ -82,6 +82,7 @@ trait DefaultReadWriteTest extends TempDirectory { self: Suite =>
* - Explicitly set Params, and train model
* - Test save/load using [[testDefaultReadWrite()]] on Estimator and Model
* - Check Params on Estimator and Model
+ * - Compare model data
*
* This requires that the [[Estimator]] and [[Model]] share the same set of [[Param]]s.
* @param estimator Estimator to test
@@ -117,6 +118,8 @@ trait DefaultReadWriteTest extends TempDirectory { self: Suite =>
val param = model.getParam(p)
assert(model.get(param).get === model2.get(param).get)
}
+
+ checkModelData(model, model2)
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org