You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by evanyc15 <gi...@git.apache.org> on 2015/10/02 21:45:24 UTC
[GitHub] spark pull request: [SPARK-10779][PYSPARK][MLLIB] Set initialModel...
GitHub user evanyc15 opened a pull request:
https://github.com/apache/spark/pull/8967
[SPARK-10779][PYSPARK][MLLIB] Set initialModel for KMeans model in PySpark (spark.mllib)
Provide initialModel param for pyspark.mllib.clustering.KMeans
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/evanyc15/spark SPARK-10779-pyspark-mllib
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/8967.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #8967
----
commit 3287f2a52410357cb75064e03d2a39e2eb1317d3
Author: Evan Chen <ch...@us.ibm.com>
Date: 2015-09-30T01:17:09Z
Modified PythonMMLLibAPI to set initialModel for KMeansModel in PySpark
commit ce2e972bbfb1fd899dce4ca7b35aa486f96facce
Author: Evan Chen <ch...@us.ibm.com>
Date: 2015-09-30T20:38:09Z
Added if statements for type checking
commit 45780d3f7d335b1449528835401022fbf2b1b345
Author: Evan Chen <ch...@us.ibm.com>
Date: 2015-09-30T22:23:12Z
Working on Doctests now
commit d2380ac11c26d6848889f83321f1703e31efdbf1
Author: Evan Chen <ch...@us.ibm.com>
Date: 2015-10-02T19:36:32Z
Doctests written, ready for pull request
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10779][PYSPARK][MLLIB] Set initialModel...
Posted by jkbradley <gi...@git.apache.org>.
Github user jkbradley commented on the pull request:
https://github.com/apache/spark/pull/8967#issuecomment-146345893
Thanks, I'll go ahead and merge this, though the style still isn't quite what I wrote.... I know style is annoying to match, but it helps a lot with keeping the codebase clean and easy to read.
LGTM, merging with master
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10779][PYSPARK][MLLIB] Set initialModel...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/8967
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10779][PYSPARK][MLLIB] Set initialModel...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8967#issuecomment-146303849
[Test build #1852 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1852/console) for PR 8967 at commit [`e8062c6`](https://github.com/apache/spark/commit/e8062c6eb0ce0c3425cc4f36cbbf72bfd555e65a).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10779][PYSPARK][MLLIB] Set initialModel...
Posted by jkbradley <gi...@git.apache.org>.
Github user jkbradley commented on the pull request:
https://github.com/apache/spark/pull/8967#issuecomment-146292742
Just 2 tiny comments.
Btw, only committers can get Jenkins to run tests on demand. (This helps avoid overtaxing Jenkins.)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10779][PYSPARK][MLLIB] Set initialModel...
Posted by jkbradley <gi...@git.apache.org>.
Github user jkbradley commented on a diff in the pull request:
https://github.com/apache/spark/pull/8967#discussion_r41430454
--- Diff: python/pyspark/mllib/clustering.py ---
@@ -144,10 +150,16 @@ class KMeans(object):
@classmethod
def train(cls, rdd, k, maxIterations=100, runs=1, initializationMode="k-means||",
- seed=None, initializationSteps=5, epsilon=1e-4):
+ seed=None, initializationSteps=5, epsilon=1e-4, initialModel=None):
"""Train a k-means clustering model."""
+ clusterInitialModel = []
+ if initialModel is not None:
+ if not isinstance(initialModel, KMeansModel):
+ raise Exception("initialModel needs to be of type KMeansModel")
--- End diff --
Please state what the bad type is in the error: ```type(initialModel)```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10779][PYSPARK][MLLIB] Set initialModel...
Posted by evanyc15 <gi...@git.apache.org>.
Github user evanyc15 commented on the pull request:
https://github.com/apache/spark/pull/8967#issuecomment-146021824
Hey jkbradley,
I have pushed another commit that removes the excess parameters in the Doctest and adds an if statement for checking if initialModel is not None and not isinstance of KMeansModel
Thanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10779][PYSPARK][MLLIB] Set initialModel...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8967#issuecomment-146073431
[Test build #1849 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1849/consoleFull) for PR 8967 at commit [`fdde001`](https://github.com/apache/spark/commit/fdde0010442957dd152fb18b963958f9bffc3ece).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10779][PYSPARK][MLLIB] Set initialModel...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8967#issuecomment-145138721
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10779][PYSPARK][MLLIB] Set initialModel...
Posted by jkbradley <gi...@git.apache.org>.
Github user jkbradley commented on the pull request:
https://github.com/apache/spark/pull/8967#issuecomment-146005437
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10779][PYSPARK][MLLIB] Set initialModel...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8967#issuecomment-146291615
[Test build #1852 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1852/consoleFull) for PR 8967 at commit [`e8062c6`](https://github.com/apache/spark/commit/e8062c6eb0ce0c3425cc4f36cbbf72bfd555e65a).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10779][PYSPARK][MLLIB] Set initialModel...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8967#issuecomment-146077350
[Test build #1849 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1849/console) for PR 8967 at commit [`fdde001`](https://github.com/apache/spark/commit/fdde0010442957dd152fb18b963958f9bffc3ece).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10779][PYSPARK][MLLIB] Set initialModel...
Posted by evanyc15 <gi...@git.apache.org>.
Github user evanyc15 commented on the pull request:
https://github.com/apache/spark/pull/8967#issuecomment-146285890
Jenkins test this
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10779][PYSPARK][MLLIB] Set initialModel...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8967#issuecomment-146016265
[Test build #1846 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1846/console) for PR 8967 at commit [`d2380ac`](https://github.com/apache/spark/commit/d2380ac11c26d6848889f83321f1703e31efdbf1).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10779][PYSPARK][MLLIB] Set initialModel...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8967#issuecomment-146328555
[Test build #1853 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1853/consoleFull) for PR 8967 at commit [`1cb872d`](https://github.com/apache/spark/commit/1cb872de09c94d82ca6e7a646cef05d63106d47e).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10779][PYSPARK][MLLIB] Set initialModel...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8967#issuecomment-146340890
[Test build #1853 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1853/console) for PR 8967 at commit [`1cb872d`](https://github.com/apache/spark/commit/1cb872de09c94d82ca6e7a646cef05d63106d47e).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10779][PYSPARK][MLLIB] Set initialModel...
Posted by jkbradley <gi...@git.apache.org>.
Github user jkbradley commented on the pull request:
https://github.com/apache/spark/pull/8967#issuecomment-146006556
This looks fine other than those 2 comments. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10779][PYSPARK][MLLIB] Set initialModel...
Posted by jkbradley <gi...@git.apache.org>.
Github user jkbradley commented on a diff in the pull request:
https://github.com/apache/spark/pull/8967#discussion_r41349993
--- Diff: python/pyspark/mllib/clustering.py ---
@@ -90,6 +90,14 @@ class KMeansModel(Saveable, Loader):
... rmtree(path)
... except OSError:
... pass
+
+ >>> data = array([-383.1, 31.2, 32.2, 367.3]).reshape(2, 2)
+ >>> model = KMeans.train(
+ ... sc.parallelize(data), 3, maxIterations=0, runs=1, initializationMode="random",
--- End diff --
By remove unnecessary parameters, I meant to remove: runs, initializationMode, initializationSteps, epsilon.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10779][PYSPARK][MLLIB] Set initialModel...
Posted by jkbradley <gi...@git.apache.org>.
Github user jkbradley commented on a diff in the pull request:
https://github.com/apache/spark/pull/8967#discussion_r41430458
--- Diff: python/pyspark/mllib/clustering.py ---
@@ -90,6 +90,12 @@ class KMeansModel(Saveable, Loader):
... rmtree(path)
... except OSError:
... pass
+
+ >>> data = array([-383.1, -382.9, 28.7, 31.2, 366.2, 367.3]).reshape(3, 2)
+ >>> model = KMeans.train(sc.parallelize(data), 3, maxIterations=0,
+ ... initialModel = KMeansModel([(-1000.0, -1000.0),(5.0, 5.0),(1000.0, 1000.0)]))
--- End diff --
nit: Python style (spacing): ```initialModel=KMeansModel([(-1000.0,-1000.0), (5.0,5.0), (1000.0,1000.0)])```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10779][PYSPARK][MLLIB] Set initialModel...
Posted by jkbradley <gi...@git.apache.org>.
Github user jkbradley commented on a diff in the pull request:
https://github.com/apache/spark/pull/8967#discussion_r41324589
--- Diff: python/pyspark/mllib/clustering.py ---
@@ -90,6 +90,14 @@ class KMeansModel(Saveable, Loader):
... rmtree(path)
... except OSError:
... pass
+
+ >>> data = array([-383.1,-382.9, 28.7,31.2, 366.2,367.3]).reshape(3, 2)
+ >>> model = KMeans.train(
+ ... sc.parallelize(data), 3, maxIterations=0, runs=1, initializationMode="random",
+ ... seed=50, initializationSteps=5, epsilon=1e-4,
--- End diff --
No need to set many of these parameters for this test; remove unnecessary ones.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10779][PYSPARK][MLLIB] Set initialModel...
Posted by jkbradley <gi...@git.apache.org>.
Github user jkbradley commented on a diff in the pull request:
https://github.com/apache/spark/pull/8967#discussion_r41349992
--- Diff: python/pyspark/mllib/clustering.py ---
@@ -90,6 +90,14 @@ class KMeansModel(Saveable, Loader):
... rmtree(path)
... except OSError:
... pass
+
+ >>> data = array([-383.1, 31.2, 32.2, 367.3]).reshape(2, 2)
--- End diff --
I would recommend keeping the original data structure; I could imagine us later adding checks which print warnings when given 2 data points but 3 cluster centers.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10779][PYSPARK][MLLIB] Set initialModel...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8967#issuecomment-146006970
[Test build #1846 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1846/consoleFull) for PR 8967 at commit [`d2380ac`](https://github.com/apache/spark/commit/d2380ac11c26d6848889f83321f1703e31efdbf1).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10779][PYSPARK][MLLIB] Set initialModel...
Posted by jkbradley <gi...@git.apache.org>.
Github user jkbradley commented on a diff in the pull request:
https://github.com/apache/spark/pull/8967#discussion_r41324593
--- Diff: python/pyspark/mllib/clustering.py ---
@@ -144,10 +152,14 @@ class KMeans(object):
@classmethod
def train(cls, rdd, k, maxIterations=100, runs=1, initializationMode="k-means||",
- seed=None, initializationSteps=5, epsilon=1e-4):
+ seed=None, initializationSteps=5, epsilon=1e-4, initialModel=None):
"""Train a k-means clustering model."""
+ cluster_initialModel = []
+ if initialModel is not None and isinstance(initialModel, KMeansModel):
--- End diff --
Raise error if not None but not instance of KMeansModel
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10779][PYSPARK][MLLIB] Set initialModel...
Posted by evanyc15 <gi...@git.apache.org>.
Github user evanyc15 commented on the pull request:
https://github.com/apache/spark/pull/8967#issuecomment-146029714
Jenkins test this
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10779][PYSPARK][MLLIB] Set initialModel...
Posted by jkbradley <gi...@git.apache.org>.
Github user jkbradley commented on a diff in the pull request:
https://github.com/apache/spark/pull/8967#discussion_r41349995
--- Diff: python/pyspark/mllib/clustering.py ---
@@ -144,10 +152,16 @@ class KMeans(object):
@classmethod
def train(cls, rdd, k, maxIterations=100, runs=1, initializationMode="k-means||",
- seed=None, initializationSteps=5, epsilon=1e-4):
+ seed=None, initializationSteps=5, epsilon=1e-4, initialModel=None):
"""Train a k-means clustering model."""
+ clusterInitialModel = []
+ if initialModel is not None and not isinstance(initialModel, KMeansModel):
--- End diff --
Can you please use nested if statements to reduce code duplication?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org