You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Apache Spark (JIRA)" <ji...@apache.org> on 2015/06/02 07:14:17 UTC
[jira] [Commented] (SPARK-7432) Flaky test in PySpark
CrossValidator doc test
[ https://issues.apache.org/jira/browse/SPARK-7432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14568535#comment-14568535 ]
Apache Spark commented on SPARK-7432:
-------------------------------------
User 'mengxr' has created a pull request for this issue:
https://github.com/apache/spark/pull/6572
> Flaky test in PySpark CrossValidator doc test
> ---------------------------------------------
>
> Key: SPARK-7432
> URL: https://issues.apache.org/jira/browse/SPARK-7432
> Project: Spark
> Issue Type: Bug
> Components: ML, PySpark
> Affects Versions: 1.4.0
> Reporter: Joseph K. Bradley
> Assignee: Xiangrui Meng
> Priority: Critical
>
> There was a test failure in the doc test in Python CrossValidator:
> [https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32058/consoleFull]
> Here's the full doc test:
> {code}
> >>> from pyspark.ml.classification import LogisticRegression
> >>> from pyspark.ml.evaluation import BinaryClassificationEvaluator
> >>> from pyspark.mllib.linalg import Vectors
> >>> dataset = sqlContext.createDataFrame(
> ... [(Vectors.dense([0.0, 1.0]), 0.0),
> ... (Vectors.dense([1.0, 2.0]), 1.0),
> ... (Vectors.dense([0.55, 3.0]), 0.0),
> ... (Vectors.dense([0.45, 4.0]), 1.0),
> ... (Vectors.dense([0.51, 5.0]), 1.0)] * 10,
> ... ["features", "label"])
> >>> lr = LogisticRegression()
> >>> grid = ParamGridBuilder().addGrid(lr.maxIter, [0, 1, 5]).build()
> >>> evaluator = BinaryClassificationEvaluator()
> >>> cv = CrossValidator(estimator=lr, estimatorParamMaps=grid, evaluator=evaluator)
> >>> cvModel = cv.fit(dataset)
> >>> expected = lr.fit(dataset, {lr.maxIter: 5}).transform(dataset)
> >>> cvModel.transform(dataset).collect() == expected.collect()
> True
> {code}
> Here's the failure message:
> {code}
> Running test: pyspark/ml/tuning.py ... **********************************************************************
> File "pyspark/ml/tuning.py", line 108, in __main__.CrossValidator
> Failed example:
> cvModel.transform(dataset).collect() == expected.collect()
> Expected:
> True
> Got:
> False
> **********************************************************************
> 1 of 11 in __main__.CrossValidator
> ***Test Failed*** 1 failures.
> Had test failures; see logs.
> [error] Got a return code of 255 on line 240 of the run-tests script.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org