You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by MLnick <gi...@git.apache.org> on 2016/06/30 13:51:26 UTC
[GitHub] spark pull request #13997: [SPAR-16328K][ML][MLLIB][PYSPARK] Add 'asML' and ...
GitHub user MLnick opened a pull request:
https://github.com/apache/spark/pull/13997
[SPAR-16328K][ML][MLLIB][PYSPARK] Add 'asML' and 'fromML' conversion methods to PySpark linalg
The move to `ml.linalg` created `asML`/`fromML` utility methods in Scala/Java for converting between representations. These are missing in Python, this PR adds them.
## How was this patch tested?
New doctests.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/MLnick/spark SPARK-16328-python-linalg-convert
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/13997.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #13997
----
commit 35974c9bd813c28e591197bcf4d069c70b834fe4
Author: Nick Pentreath <ni...@za.ibm.com>
Date: 2016-06-30T13:47:56Z
Add 'asML' and 'fromML' conversion methods to PySpark linalg
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #13997: [SPARK-16328][ML][MLLIB][PYSPARK] Add 'asML' and ...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/13997
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #13997: [SPARK-16328][ML][MLLIB][PYSPARK] Add 'asML' and 'fromML...
Posted by jkbradley <gi...@git.apache.org>.
Github user jkbradley commented on the issue:
https://github.com/apache/spark/pull/13997
Thanks for the updates! For tests, I'm all for including some doc tests in Python if they provide good examples for the docs. But developers, myself included, have tended to include too much in doc tests in the past.
LGTM
Merging with master and branch-2.0
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #13997: [SPARK-16328][ML][MLLIB][PYSPARK] Add 'asML' and ...
Posted by jkbradley <gi...@git.apache.org>.
Github user jkbradley commented on a diff in the pull request:
https://github.com/apache/spark/pull/13997#discussion_r69203781
--- Diff: python/pyspark/mllib/linalg/__init__.py ---
@@ -1236,6 +1359,44 @@ def sparse(numRows, numCols, colPtrs, rowIndices, values):
"""
return SparseMatrix(numRows, numCols, colPtrs, rowIndices, values)
+ @staticmethod
+ def fromML(mat):
+ """
+ Convert a matrix from the new mllib-local representation.
+ This does NOT copy the data; it copies references.
+
+ >>> mllibDM1 = Matrices.dense(2, 2, [1, 2, 3, 4])
+ >>> mlDM = newlinalg.Matrices.dense(2, 2, [1, 2, 3, 4])
+ >>> mllibDM2 = Matrices.fromML(mlDM)
+ >>> mllibDM1 == mllibDM2
+ True
+ >>> mllibDMt1 = DenseMatrix(2, 2, [1, 2, 3, 4], True)
+ >>> mlDMt = newlinalg.DenseMatrix(2, 2, [1, 2, 3, 4], True)
+ >>> mllibDMt2 = Matrices.fromML(mlDMt)
+ >>> mllibDMt1 == mllibDMt2
+ True
+ >>> mllibSM1 = Matrices.sparse(2, 2, [0, 2, 3], [0, 1, 1], [2, 3, 4])
+ >>> mlSM = newlinalg.Matrices.sparse(2, 2, [0, 2, 3], [0, 1, 1], [2, 3, 4])
+ >>> mllibSM2 = Matrices.fromML(mlSM)
+ >>> mllibSM1 == mllibSM2
+ True
+ >>> mllibSMt1 = SparseMatrix(2, 2, [0, 2, 3], [0, 1, 1], [2, 3, 4], True)
+ >>> mlSMt = newlinalg.SparseMatrix(2, 2, [0, 2, 3], [0, 1, 1], [2, 3, 4], True)
+ >>> mllibSMt2 = Matrices.fromML(mlSMt)
+ >>> mllibSMt1 == mllibSMt2
+ True
+
+ :param vec: a :py:class:`pyspark.ml.linalg.Matrix`
+ :return: a :py:class:`pyspark.mllib.linalg.Matrix`
--- End diff --
versionadded
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #13997: [SPARK-16328][ML][MLLIB][PYSPARK] Add 'asML' and 'fromML...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13997
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #13997: [SPARK-16328][ML][MLLIB][PYSPARK] Add 'asML' and ...
Posted by MechCoder <gi...@git.apache.org>.
Github user MechCoder commented on a diff in the pull request:
https://github.com/apache/spark/pull/13997#discussion_r69176457
--- Diff: python/pyspark/mllib/linalg/__init__.py ---
@@ -846,6 +890,33 @@ def dense(*elements):
return DenseVector(elements)
@staticmethod
+ def fromML(vec):
+ """
+ Convert a vector from the new mllib-local representation.
+ This does NOT copy the data; it copies references.
+
+ >>> mllibDV1 = Vectors.dense([1, 2, 3])
+ >>> mlDV = newlinalg.Vectors.dense([1, 2, 3])
+ >>> mllibDV2 = Vectors.fromML(mlDV)
+ >>> mllibDV1 == mllibDV2
+ True
+ >>> mllibSV1 = Vectors.sparse(4, {1: 1.0, 3: 5.5})
+ >>> mlSV = newlinalg.Vectors.sparse(4, {1: 1.0, 3: 5.5})
+ >>> mllibSV2 = Vectors.fromML(mlSV)
+ >>> mllibSV1 == mllibSV2
+ True
+
+ :param vec: a :py:class:`pyspark.ml.linalg.Vector`
+ :return: a :py:class:`pyspark.mllib.linalg.Vector`
+ """
+ if type(vec) == newlinalg.DenseVector:
--- End diff --
It's a common pythonic practise to use `isinstance` in such cases. If we inherit something from `DenseVector`, then this check will fail.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #13997: [SPARK-16328][ML][MLLIB][PYSPARK] Add 'asML' and 'fromML...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13997
**[Test build #61544 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61544/consoleFull)** for PR 13997 at commit [`35974c9`](https://github.com/apache/spark/commit/35974c9bd813c28e591197bcf4d069c70b834fe4).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #13997: [SPARK-16328][ML][MLLIB][PYSPARK] Add 'asML' and 'fromML...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13997
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61544/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #13997: [SPARK-16328][ML][MLLIB][PYSPARK] Add 'asML' and 'fromML...
Posted by jkbradley <gi...@git.apache.org>.
Github user jkbradley commented on the issue:
https://github.com/apache/spark/pull/13997
Looks good, except for a few minor comments
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #13997: [SPARK-16328][ML][MLLIB][PYSPARK] Add 'asML' and 'fromML...
Posted by MechCoder <gi...@git.apache.org>.
Github user MechCoder commented on the issue:
https://github.com/apache/spark/pull/13997
LGTM pending nitpicks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #13997: [SPARK-16328][ML][MLLIB][PYSPARK] Add 'asML' and ...
Posted by jkbradley <gi...@git.apache.org>.
Github user jkbradley commented on a diff in the pull request:
https://github.com/apache/spark/pull/13997#discussion_r69203766
--- Diff: python/pyspark/mllib/linalg/__init__.py ---
@@ -846,6 +890,33 @@ def dense(*elements):
return DenseVector(elements)
@staticmethod
+ def fromML(vec):
+ """
+ Convert a vector from the new mllib-local representation.
+ This does NOT copy the data; it copies references.
+
+ >>> mllibDV1 = Vectors.dense([1, 2, 3])
+ >>> mlDV = newlinalg.Vectors.dense([1, 2, 3])
+ >>> mllibDV2 = Vectors.fromML(mlDV)
+ >>> mllibDV1 == mllibDV2
+ True
+ >>> mllibSV1 = Vectors.sparse(4, {1: 1.0, 3: 5.5})
+ >>> mlSV = newlinalg.Vectors.sparse(4, {1: 1.0, 3: 5.5})
+ >>> mllibSV2 = Vectors.fromML(mlSV)
+ >>> mllibSV1 == mllibSV2
+ True
+
+ :param vec: a :py:class:`pyspark.ml.linalg.Vector`
+ :return: a :py:class:`pyspark.mllib.linalg.Vector`
--- End diff --
versionadded
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #13997: [SPARK-16328][ML][MLLIB][PYSPARK] Add 'asML' and 'fromML...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13997
**[Test build #61572 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61572/consoleFull)** for PR 13997 at commit [`05ff527`](https://github.com/apache/spark/commit/05ff5274c3562d4c5994960b835af88c836d6c8a).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #13997: [SPARK-16328][ML][MLLIB][PYSPARK] Add 'asML' and 'fromML...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13997
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61556/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #13997: [SPARK-16328][ML][MLLIB][PYSPARK] Add 'asML' and 'fromML...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13997
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61572/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #13997: [SPARK-16328][ML][MLLIB][PYSPARK] Add 'asML' and 'fromML...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13997
**[Test build #61572 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61572/consoleFull)** for PR 13997 at commit [`05ff527`](https://github.com/apache/spark/commit/05ff5274c3562d4c5994960b835af88c836d6c8a).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #13997: [SPARK-16328][ML][MLLIB][PYSPARK] Add 'asML' and ...
Posted by jkbradley <gi...@git.apache.org>.
Github user jkbradley commented on a diff in the pull request:
https://github.com/apache/spark/pull/13997#discussion_r69204142
--- Diff: python/pyspark/mllib/linalg/__init__.py ---
@@ -1236,6 +1359,44 @@ def sparse(numRows, numCols, colPtrs, rowIndices, values):
"""
return SparseMatrix(numRows, numCols, colPtrs, rowIndices, values)
+ @staticmethod
+ def fromML(mat):
+ """
+ Convert a matrix from the new mllib-local representation.
+ This does NOT copy the data; it copies references.
+
+ >>> mllibDM1 = Matrices.dense(2, 2, [1, 2, 3, 4])
--- End diff --
Btw, in the future, I'd like us to treat doc tests more like documentation and put unit tests within tests.py. But this isn't a big deal and I'm OK with leaving your doc tests as they are for now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #13997: [SPARK-16328][ML][MLLIB][PYSPARK] Add 'asML' and ...
Posted by MechCoder <gi...@git.apache.org>.
Github user MechCoder commented on a diff in the pull request:
https://github.com/apache/spark/pull/13997#discussion_r69176771
--- Diff: python/pyspark/mllib/linalg/__init__.py ---
@@ -1044,6 +1122,28 @@ def toSparse(self):
return SparseMatrix(self.numRows, self.numCols, colPtrs, rowIndices, values)
+ def asML(self):
+ """
+ Convert this matrix to the new mllib-local representation.
+ This does NOT copy the data; it copies references.
+
+ >>> mllibDM = Matrices.dense(2, 2, [0, 1, 2, 3])
+ >>> mlDM1 = newlinalg.Matrices.dense(2, 2, [0, 1, 2, 3])
+ >>> mlDM2 = mllibDM.asML()
+ >>> mlDM2 == mlDM1
+ True
+ >>> mllibDMt = DenseMatrix(2, 2, [0, 1, 2, 3], True)
+ >>> mlDMt1 = newlinalg.DenseMatrix(2, 2, [0, 1, 2, 3], True)
+ >>> mlDMt2 = mllibDMt.asML()
+ >>> mlDMt2 == mlDMt1
+ True
+
+ :return: :py:class:`pyspark.ml.linalg.DenseMatrix`
+
+ .. versionadded:: 2.0.0
+ """
+ return newlinalg.DenseMatrix(self.numRows, self.numCols, self.values, self.isTransposed)
--- End diff --
> 79 ;)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #13997: [SPARK-16328][ML][MLLIB][PYSPARK] Add 'asML' and ...
Posted by jkbradley <gi...@git.apache.org>.
Github user jkbradley commented on a diff in the pull request:
https://github.com/apache/spark/pull/13997#discussion_r69203849
--- Diff: python/pyspark/mllib/linalg/__init__.py ---
@@ -1236,6 +1359,44 @@ def sparse(numRows, numCols, colPtrs, rowIndices, values):
"""
return SparseMatrix(numRows, numCols, colPtrs, rowIndices, values)
+ @staticmethod
+ def fromML(mat):
+ """
+ Convert a matrix from the new mllib-local representation.
+ This does NOT copy the data; it copies references.
+
+ >>> mllibDM1 = Matrices.dense(2, 2, [1, 2, 3, 4])
+ >>> mlDM = newlinalg.Matrices.dense(2, 2, [1, 2, 3, 4])
+ >>> mllibDM2 = Matrices.fromML(mlDM)
+ >>> mllibDM1 == mllibDM2
+ True
+ >>> mllibDMt1 = DenseMatrix(2, 2, [1, 2, 3, 4], True)
+ >>> mlDMt = newlinalg.DenseMatrix(2, 2, [1, 2, 3, 4], True)
+ >>> mllibDMt2 = Matrices.fromML(mlDMt)
+ >>> mllibDMt1 == mllibDMt2
+ True
+ >>> mllibSM1 = Matrices.sparse(2, 2, [0, 2, 3], [0, 1, 1], [2, 3, 4])
+ >>> mlSM = newlinalg.Matrices.sparse(2, 2, [0, 2, 3], [0, 1, 1], [2, 3, 4])
+ >>> mllibSM2 = Matrices.fromML(mlSM)
+ >>> mllibSM1 == mllibSM2
+ True
+ >>> mllibSMt1 = SparseMatrix(2, 2, [0, 2, 3], [0, 1, 1], [2, 3, 4], True)
+ >>> mlSMt = newlinalg.SparseMatrix(2, 2, [0, 2, 3], [0, 1, 1], [2, 3, 4], True)
+ >>> mllibSMt2 = Matrices.fromML(mlSMt)
+ >>> mllibSMt1 == mllibSMt2
+ True
+
+ :param vec: a :py:class:`pyspark.ml.linalg.Matrix`
--- End diff --
vec -> mat
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #13997: [SPARK-16328][ML][MLLIB][PYSPARK] Add 'asML' and 'fromML...
Posted by MLnick <gi...@git.apache.org>.
Github user MLnick commented on the issue:
https://github.com/apache/spark/pull/13997
@jkbradley fair point for the tests - I actually just moved them to `tests.py` since it's just a copy-paste
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #13997: [SPARK-16328][ML][MLLIB][PYSPARK] Add 'asML' and 'fromML...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13997
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #13997: [SPAR-16328K][ML][MLLIB][PYSPARK] Add 'asML' and 'fromML...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13997
**[Test build #61544 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61544/consoleFull)** for PR 13997 at commit [`35974c9`](https://github.com/apache/spark/commit/35974c9bd813c28e591197bcf4d069c70b834fe4).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #13997: [SPARK-16328][ML][MLLIB][PYSPARK] Add 'asML' and ...
Posted by MLnick <gi...@git.apache.org>.
Github user MLnick commented on a diff in the pull request:
https://github.com/apache/spark/pull/13997#discussion_r69178051
--- Diff: python/pyspark/mllib/linalg/__init__.py ---
@@ -846,6 +890,33 @@ def dense(*elements):
return DenseVector(elements)
@staticmethod
+ def fromML(vec):
+ """
+ Convert a vector from the new mllib-local representation.
+ This does NOT copy the data; it copies references.
+
+ >>> mllibDV1 = Vectors.dense([1, 2, 3])
+ >>> mlDV = newlinalg.Vectors.dense([1, 2, 3])
+ >>> mllibDV2 = Vectors.fromML(mlDV)
+ >>> mllibDV1 == mllibDV2
+ True
+ >>> mllibSV1 = Vectors.sparse(4, {1: 1.0, 3: 5.5})
+ >>> mlSV = newlinalg.Vectors.sparse(4, {1: 1.0, 3: 5.5})
+ >>> mllibSV2 = Vectors.fromML(mlSV)
+ >>> mllibSV1 == mllibSV2
+ True
+
+ :param vec: a :py:class:`pyspark.ml.linalg.Vector`
+ :return: a :py:class:`pyspark.mllib.linalg.Vector`
+ """
+ if type(vec) == newlinalg.DenseVector:
--- End diff --
doh! yeah of course, wasn't thinking. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #13997: [SPARK-16328][ML][MLLIB][PYSPARK] Add 'asML' and 'fromML...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13997
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #13997: [SPARK-16328][ML][MLLIB][PYSPARK] Add 'asML' and 'fromML...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13997
**[Test build #61556 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61556/consoleFull)** for PR 13997 at commit [`a180421`](https://github.com/apache/spark/commit/a1804213bd1cb9b26e2693826e44d548341b8942).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #13997: [SPARK-16328][ML][MLLIB][PYSPARK] Add 'asML' and 'fromML...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13997
**[Test build #61556 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61556/consoleFull)** for PR 13997 at commit [`a180421`](https://github.com/apache/spark/commit/a1804213bd1cb9b26e2693826e44d548341b8942).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #13997: [SPAR-16328K][ML][MLLIB][PYSPARK] Add 'asML' and 'fromML...
Posted by MLnick <gi...@git.apache.org>.
Github user MLnick commented on the issue:
https://github.com/apache/spark/pull/13997
cc @jkbradley @mengxr @yanboliang
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org