You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by brkyvz <gi...@git.apache.org> on 2014/09/05 23:04:47 UTC
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
GitHub user brkyvz opened a pull request:
https://github.com/apache/spark/pull/2294
[SPARK-3418] Sparse Matrix support (CCS) and additional native BLAS operations added
Local `SparseMatrix` support added in Compressed Column Storage (CCS) format in addition to Level-2 and Level-3 BLAS operations such as dgemv and dgemm respectively.
BLAS doesn't support sparse matrix operations, therefore support for `SparseMatrix`-`DenseMatrix` multiplication and `SparseMatrix`-`DenseVector` implementations have been added. I will post performance comparisons in the comments momentarily.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/brkyvz/spark SPARK-3418
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/2294.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2294
----
commit 5138d3f220efb8ec4cc7fc944112497ab0c8c50c
Author: Burak <br...@gmail.com>
Date: 2014-09-05T20:34:24Z
[SPARK-3418][MLlib] Sparse Matrix support and additional native BLAS operations added
commit 4362ff1d6b79604b9919da1d550ed466021b8f5c
Author: Burak <br...@gmail.com>
Date: 2014-09-05T20:51:45Z
[SPARK-3418][MLlib] Matrix unit tests expanded with indexing and updating
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17255310
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala ---
@@ -63,7 +88,162 @@ class DenseMatrix(val numRows: Int, val numCols: Int, val values: Array[Double])
override def toArray: Array[Double] = values
- private[mllib] override def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+ private [mllib] def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+
+ private [mllib] def apply(i: Int): Double = values(i)
+
+ private [mllib] def apply(r: Int, c: Int): Double = values(index(r, c))
+
+ private [mllib] def index(r: Int, c: Int): Int = r + numRows * c
+
+ private [mllib] def update(r: Int, c: Int, v: Double){
+ values(index(r, c)) = v
+ }
+
+ def copy = new DenseMatrix(numRows, numCols, values.clone())
+}
+
+/**
+ * Factory methods for [[org.apache.spark.mllib.linalg.DenseMatrix]].
+ *
+ * These methods can be used to generate common matrix types such as the Identity matrix, any
+ * diagonal matrix, zero matrix, and random matrices.
+ */
+object DenseMatrix {
+
+ /**
+ * Generate a `DenseMatrix` consisting of zeros.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values of zeros
+ */
+ def zeros(numRows: Int, numCols: Int) =
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(0.0))
+
+ /**
+ * Generate a `DenseMatrix` consisting of ones.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values of ones
+ */
+ def ones(numRows: Int, numCols: Int) =
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(1.0))
+
+ /**
+ * Generate an Identity Matrix in `DenseMatrix` format.
+ * @param n number of rows and columns of the matrix
+ * @return `DenseMatrix` with size `n` x `n` and values of ones on the diagonal
+ */
+ def eye(n: Int) = {
+ val identity = DenseMatrix.zeros(n,n)
--- End diff --
space after `,`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17255320
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala ---
@@ -63,7 +88,162 @@ class DenseMatrix(val numRows: Int, val numCols: Int, val values: Array[Double])
override def toArray: Array[Double] = values
- private[mllib] override def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+ private [mllib] def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+
+ private [mllib] def apply(i: Int): Double = values(i)
+
+ private [mllib] def apply(r: Int, c: Int): Double = values(index(r, c))
+
+ private [mllib] def index(r: Int, c: Int): Int = r + numRows * c
+
+ private [mllib] def update(r: Int, c: Int, v: Double){
+ values(index(r, c)) = v
+ }
+
+ def copy = new DenseMatrix(numRows, numCols, values.clone())
+}
+
+/**
+ * Factory methods for [[org.apache.spark.mllib.linalg.DenseMatrix]].
+ *
+ * These methods can be used to generate common matrix types such as the Identity matrix, any
+ * diagonal matrix, zero matrix, and random matrices.
+ */
+object DenseMatrix {
+
+ /**
+ * Generate a `DenseMatrix` consisting of zeros.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values of zeros
+ */
+ def zeros(numRows: Int, numCols: Int) =
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(0.0))
+
+ /**
+ * Generate a `DenseMatrix` consisting of ones.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values of ones
+ */
+ def ones(numRows: Int, numCols: Int) =
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(1.0))
+
+ /**
+ * Generate an Identity Matrix in `DenseMatrix` format.
+ * @param n number of rows and columns of the matrix
+ * @return `DenseMatrix` with size `n` x `n` and values of ones on the diagonal
+ */
+ def eye(n: Int) = {
+ val identity = DenseMatrix.zeros(n,n)
+ for (i <- 0 until n){
+ identity.update(i, i, 1.0)
+ }
+ identity
+ }
+
+ /**
+ * Generate a `DenseMatrix` consisting of i.i.d. uniform random numbers.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values in U(0, 1)
+ */
+ def rand(numRows: Int, numCols: Int) = {
+ val rand = new scala.util.Random
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(rand.nextDouble()))
+ }
+
+ /**
+ * Generate a `DenseMatrix` consisting of i.i.d. gaussian random numbers.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values in N(0, 1)
+ */
+ def randn(numRows: Int, numCols: Int) = {
+ val rand = new scala.util.Random
--- End diff --
ditto
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17709063
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +201,368 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ if (alpha == 0.0) {
+ logWarning("gemm: alpha is equal to 0. Returning C.")
+ } else {
+ A match {
+ case sparse: SparseMatrix =>
+ gemm(transA, transB, alpha, sparse, B, beta, C)
+ case dense: DenseMatrix =>
+ gemm(transA, transB, alpha, dense, B, beta, C)
+ case _ =>
+ throw new IllegalArgumentException(s"gemm doesn't support matrix type ${A.getClass}.")
+ }
+ }
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ gemm(false, false, alpha, A, B, beta, C)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `DenseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: DenseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+ val tAstr = if (!transA) "N" else "T"
+ val tBstr = if (!transB) "N" else "T"
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ nativeBLAS.dgemm(tAstr, tBstr, mA, nB, kA, alpha, A.values, A.numRows, B.values, B.numRows,
+ beta, C.values, C.numRows)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `SparseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: SparseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ val Avals = A.values
+ val Arows = if (!transA) A.rowIndices else A.colPtrs
+ val Acols = if (!transA) A.colPtrs else A.rowIndices
+
+ // Slicing is easy in this case. This is the optimal multiplication setting for sparse matrices
+ if (transA){
+ var colCounterForB = 0
+ if (!transB){ // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var rowCounterForA = 0
+ val Cstart = colCounterForB * mA
+ val Bstart = colCounterForB * kA
+ while (rowCounterForA < mA) {
+ var i = Arows(rowCounterForA)
+ val indEnd = Arows(rowCounterForA + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B.values(Bstart + Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounterForA
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ } else {
+ while (colCounterForB < nB) {
+ var rowCounter = 0
+ val Cstart = colCounterForB * mA
+ while (rowCounter < mA) {
+ var i = Arows(rowCounter)
+ val indEnd = Arows(rowCounter + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B(colCounterForB, Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounter
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounter += 1
+ }
+ colCounterForB += 1
+ }
+ }
+ } else {
+ // Scale matrix first if `beta` is not equal to 0.0
+ if (beta != 0.0){
+ nativeBLAS.dscal(C.values.length, beta, C.values, 1)
+ }
+ // Perform matrix multiplication and add to C. The rows of A are multiplied by the columns of
+ // B, and added to C.
+ var colCounterForB = 0 // the column to be updated in C
+ if (!transB) { // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var colCounterForA = 0 // The column of A to multiply with the row of B
+ while (colCounterForA < kA){
+ var i = Acols(colCounterForA)
+ val indEnd = Acols(colCounterForA + 1)
+ val Bval = B(colCounterForA, colCounterForB)
+ val Cstart = colCounterForB * mA
+ while (i < indEnd){
+ C.values(Cstart + Arows(i)) += Avals(i) * Bval * alpha
+ i += 1
+ }
+ colCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ } else {
+ while (colCounterForB < nB) {
+ var colCounterForA = 0 // The column of A to multiply with the row of B
+ while (colCounterForA < kA){
+ var i = Acols(colCounterForA)
+ val indEnd = Acols(colCounterForA + 1)
+ val Bval = B(colCounterForB, colCounterForA)
+ val Cstart = colCounterForB * mA
+ while (i < indEnd){
+ C.values(Cstart + Arows(i)) += Avals(i) * Bval * alpha
+ i += 1
+ }
+ colCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ }
+ }
+ }
+
+ /**
+ * y := alpha * A * x + beta * y
+ * @param trans specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ * @param beta a scalar that can be used to scale vector y.
+ * @param y the resulting vector y. Size of m x 1.
+ */
+ def gemv(
+ trans: Boolean,
+ alpha: Double,
+ A: Matrix,
+ x: DenseVector,
+ beta: Double,
+ y: DenseVector): Unit = {
+
+ val mA: Int = if (!trans) A.numRows else A.numCols
+ val nx: Int = x.size
+ val nA: Int = if (!trans) A.numCols else A.numRows
+
+ require(nA == nx, s"The columns of A don't match the number of elements of x. A: $nA, x: $nx")
+ require(mA == y.size,
+ s"The rows of A don't match the number of elements of y. A: $mA, y:${y.size}}")
+ if (alpha == 0.0) {
+ logWarning("gemv: alpha is equal to 0. Returning y.")
+ } else {
+ A match {
+ case sparse: SparseMatrix =>
+ gemv(trans, alpha, sparse, x, beta, y)
+ case dense: DenseMatrix =>
+ gemv(trans, alpha, dense, x, beta, y)
+ case _ =>
+ throw new IllegalArgumentException(s"gemv doesn't support matrix type ${A.getClass}.")
+ }
+ }
+ }
+
+ /**
+ * y := alpha * A * x + beta * y
+ *
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ * @param beta a scalar that can be used to scale vector y.
+ * @param y the resulting vector y. Size of m x 1.
+ */
+ def gemv(
+ alpha: Double,
+ A: Matrix,
+ x: DenseVector,
+ beta: Double,
+ y: DenseVector): Unit = {
+ gemv(false, alpha, A, x, beta, y)
+ }
+
+ /**
+ * y := alpha * A * x
+ *
+ * @param trans specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ *
+ * @return `DenseVector` y, the result of the matrix-vector multiplication. Size of m x 1.
+ */
+ def gemv(
+ trans: Boolean,
+ alpha: Double,
+ A: Matrix,
+ x: DenseVector): DenseVector = {
+ val m = if(!trans) A.numRows else A.numCols
+
+ val y: DenseVector = new DenseVector(Array.fill(m)(0.0))
+ gemv(trans, alpha, A, x, 0.0, y)
+
+ y
+ }
+
+ /**
+ * y := alpha * A * x
+ *
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ *
+ * @return `DenseVector` y, the result of the matrix-vector multiplication. Size of m x 1.
+ */
+ def gemv(
+ alpha: Double,
+ A: Matrix,
+ x: DenseVector): DenseVector = {
--- End diff --
ditto: cleaner if we don't have methods in BLAS that allocates workspace.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by brkyvz <gi...@git.apache.org>.
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-56089214
Tests for the Level 2 operations (gemv):
nativeBLAS was faster across the board. Here are the results (in ms):
m | n | f2j | native
---|----|-----|--------
1000 | 500 | 0.7 | 0.1
1000 | 1000 | 2.3 | 0.7
1000 | 5000 | 7.7 | 3.6
5000 | 500 | 3.8 | 2.4
5000 | 1000 | 7.7 | 4.0
5000 | 5000 | 38.4 | 17.4
10000 | 500 | 6.5 | 4.5
10000 | 1000 | 13.0 | 8.9
10000 | 5000 | 75.4 | 36.7
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-54701390
[QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19889/consoleFull) for PR 2294 at commit [`56d7c85`](https://github.com/apache/spark/commit/56d7c85a1d58ef68c31208e062a4458e170111d3).
* This patch **fails** unit tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `class SparseMatrix(`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17709103
--- Diff: mllib/src/test/scala/org/apache/spark/mllib/linalg/MatricesSuite.scala ---
@@ -36,4 +36,79 @@ class MatricesSuite extends FunSuite {
Matrices.dense(3, 2, Array(0.0, 1.0, 2.0))
}
}
+
+ test("sparse matrix construction") {
+ val m = 3
+ val n = 2
+ val values = Array(1.0, 2.0, 4.0, 5.0)
+ val colIndices = Array(0, 2, 4)
+ val rowIndices = Array(1, 2, 1, 2)
+ val mat = Matrices.sparse(m, n, colIndices, rowIndices, values).asInstanceOf[SparseMatrix]
+ assert(mat.numRows === m)
+ assert(mat.numCols === n)
+ assert(mat.values.eq(values), "should not copy data")
+ }
+
+ test("sparse matrix construction with wrong number of elements") {
+ intercept[RuntimeException] {
+ Matrices.sparse(3, 2, Array(0, 1), Array(1, 2, 1), Array(0.0, 1.0, 2.0))
+ }
+
+ intercept[RuntimeException] {
+ Matrices.sparse(3, 2, Array(0, 1, 2), Array(1, 2), Array(0.0, 1.0, 2.0))
+ }
+ }
+
+ test("matrix copies are deep copies") {
+ val m = 3
+ val n = 2
+
+ val denseMat = Matrices.dense(m, n, Array(0.0, 1.0, 2.0, 3.0, 4.0, 5.0))
+ val denseCopy = denseMat.copy
+
+ assert(!denseMat.toArray.eq(denseCopy.toArray))
+
+ val values = Array(1.0, 2.0, 4.0, 5.0)
+ val colIndices = Array(0, 2, 4)
+ val rowIndices = Array(1, 2, 1, 2)
+ val sparseMat = Matrices.sparse(m, n, colIndices, rowIndices, values)
+ val sparseCopy = sparseMat.copy
+
+ assert(!sparseMat.toArray.eq(sparseCopy.toArray))
+ }
+
+ test("matrix indexing and updating") {
+ val m = 3
+ val n = 2
+ val allValues = Array(0.0, 1.0, 2.0, 3.0, 4.0, 0.0)
+
+ val denseMat = new DenseMatrix(m, n, allValues)
+
+ assert(denseMat(0, 1) == 3.0)
+ assert(denseMat(0, 1) == denseMat.values(3))
+ assert(denseMat(0, 1) == denseMat(3))
+ assert(denseMat(0, 0) == 0.0)
+
+ denseMat.update(0, 0, 10.0)
+ assert(denseMat(0, 0) == 10.0)
+ assert(denseMat.values(0) == 10.0)
+
+ val sparseValues = Array(1.0, 2.0, 3.0, 4.0)
+ val colIndices = Array(0, 2, 4)
--- End diff --
`colPtrs`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-54935726
[QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20031/consoleFull) for PR 2294 at commit [`a85ccb7`](https://github.com/apache/spark/commit/a85ccb712d83d20b178b40c32bd473d7d018a88f).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17255347
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala ---
@@ -63,7 +88,162 @@ class DenseMatrix(val numRows: Int, val numCols: Int, val values: Array[Double])
override def toArray: Array[Double] = values
- private[mllib] override def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+ private [mllib] def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+
+ private [mllib] def apply(i: Int): Double = values(i)
+
+ private [mllib] def apply(r: Int, c: Int): Double = values(index(r, c))
+
+ private [mllib] def index(r: Int, c: Int): Int = r + numRows * c
+
+ private [mllib] def update(r: Int, c: Int, v: Double){
+ values(index(r, c)) = v
+ }
+
+ def copy = new DenseMatrix(numRows, numCols, values.clone())
+}
+
+/**
+ * Factory methods for [[org.apache.spark.mllib.linalg.DenseMatrix]].
+ *
+ * These methods can be used to generate common matrix types such as the Identity matrix, any
+ * diagonal matrix, zero matrix, and random matrices.
+ */
+object DenseMatrix {
+
+ /**
+ * Generate a `DenseMatrix` consisting of zeros.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values of zeros
+ */
+ def zeros(numRows: Int, numCols: Int) =
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(0.0))
+
+ /**
+ * Generate a `DenseMatrix` consisting of ones.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values of ones
+ */
+ def ones(numRows: Int, numCols: Int) =
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(1.0))
+
+ /**
+ * Generate an Identity Matrix in `DenseMatrix` format.
+ * @param n number of rows and columns of the matrix
+ * @return `DenseMatrix` with size `n` x `n` and values of ones on the diagonal
+ */
+ def eye(n: Int) = {
+ val identity = DenseMatrix.zeros(n,n)
+ for (i <- 0 until n){
+ identity.update(i, i, 1.0)
+ }
+ identity
+ }
+
+ /**
+ * Generate a `DenseMatrix` consisting of i.i.d. uniform random numbers.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values in U(0, 1)
+ */
+ def rand(numRows: Int, numCols: Int) = {
+ val rand = new scala.util.Random
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(rand.nextDouble()))
+ }
+
+ /**
+ * Generate a `DenseMatrix` consisting of i.i.d. gaussian random numbers.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values in N(0, 1)
+ */
+ def randn(numRows: Int, numCols: Int) = {
+ val rand = new scala.util.Random
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(rand.nextGaussian()))
+ }
+
+ /**
+ * Generate a diagonal matrix in `DenseMatrix` format from the supplied values.
+ * @param values values on the diagonal of the matrix
+ * @return Square `DenseMatrix` with size `values.length` x `values.length` and `values`
+ * on the diagonal
+ */
+ def diag(values: Array[Double]) = {
+ val n = values.length
+ val matrix = DenseMatrix.eye(n)
+ for (i <- 0 until n) matrix.update(i, i, values(i))
+ matrix
+ }
+}
+
+/**
+ * Column-majored sparse matrix.
+ * The entry values are stored in Compressed-Column Storage (CCS) format.
+ * For example, the following matrix
+ * {{{
+ * 1.0 0.0 4.0
+ * 0.0 3.0 5.0
+ * 2.0 0.0 6.0
+ * }}}
+ * is stored as `values: [1.0, 2.0, 3.0, 4.0, 5.0, 6.0]`,
+ * `rowIndices=[0, 2, 1, 0, 1, 2]`, `colPointers=[0, 2, 3, 6]`.
+ *
+ * @param numRows number of rows
+ * @param numCols number of columns
+ * @param colPointers the index corresponding to the start of a new column
+ * @param rowIndices the row index of the entry
+ * @param values non-zero matrix entries in column major
+ */
+class SparseMatrix(
+ val numRows: Int,
+ val numCols: Int,
+ val colPointers: Array[Int],
+ val rowIndices: Array[Int],
+ val values: Array[Double]) extends Matrix {
+
+ require(values.length == rowIndices.length, "The number of row indices and values don't match!")
+ require(colPointers.length == numCols + 1, "The length of the column indices should be the " +
+ s"number of columns + 1. Currently, colPointers.length: ${colPointers.length}, " +
+ s"numCols: $numCols")
+
+ override def toArray: Array[Double] = values
+
+ private [mllib] def toBreeze: BM[Double] =
+ new BSM[Double](values, numRows, numCols, colPointers, rowIndices)
+
+ private [mllib] def apply(i: Int): Double = values(i)
+
+ private [mllib] def apply(r: Int, c: Int): Double = {
+ val ind = index(r,c)
+ if (ind == -1) 0.0 else values(ind)
+ }
+
+ private [mllib] def index(r: Int, c: Int): Int = {
+ val regionStart = colPointers(c)
+ val regionEnd = colPointers(c + 1)
+ val region = rowIndices.slice(regionStart, regionEnd)
+ if (region.contains(r)){
+ region.indexOf(r) + regionStart
+ } else {
+ -1 // return zero element
+ }
+ }
+
+ // TODO(Burak): Maybe convert to Breeze to update zero entries? I can't think of any MLlib
--- End diff --
I think we should not allow updating a zero entry.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-56085981
[QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20548/consoleFull) for PR 2294 at commit [`d162684`](https://github.com/apache/spark/commit/d16268496cefe06d8818b20721eede56a9de41a2).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-54698940
[QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19875/consoleFull) for PR 2294 at commit [`41b2da3`](https://github.com/apache/spark/commit/41b2da30797e3da011e3261f8c2b89b9c1608d23).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by brkyvz <gi...@git.apache.org>.
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17709785
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala ---
@@ -36,9 +37,42 @@ trait Matrix extends Serializable {
/** Converts to a breeze matrix. */
private[mllib] def toBreeze: BM[Double]
+ /** Gets the i-th element in the array backing the matrix. */
+ private[mllib] def apply(i: Int): Double
+
/** Gets the (i, j)-th element. */
- private[mllib] def apply(i: Int, j: Int): Double = toBreeze(i, j)
+ private[mllib] def apply(i: Int, j: Int): Double
+
+ /** Return the index for the (i, j)-th element in the backing array. */
+ private[mllib] def index(i: Int, j: Int): Int
+
+ /** Update element at (i, j) */
+ private[mllib] def update(i: Int, j: Int, v: Double)
+
+ /** Get a deep copy of the matrix. */
+ def copy: Matrix
+
+ /** Convenience method for `Matrix`-`DenseMatrix` multiplication. */
+ def times(y: DenseMatrix): DenseMatrix = {
+ val C: DenseMatrix = Matrices.zeros(numRows, y.numCols).asInstanceOf[DenseMatrix]
+ BLAS.gemm(false, false, 1.0, this, y, 0.0, C)
+ C
+ }
+
+ /** Convenience method for `Matrix`-`DenseVector` multiplication. */
+ def times(y: DenseVector): DenseVector = BLAS.gemv(1.0, this, y)
+
+ /** Convenience method for `Matrix`^T^-`DenseMatrix` multiplication. */
+ def transposeTimes(y: DenseMatrix): DenseMatrix = {
--- End diff --
Then we will have to implement everything (element indexing, toString, etc...) with the transpose()
check. Plus there is the binary Compatibility errors already. We may have to remove the multiply methods altogether or disable compatibility check here?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-56098271
[QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20550/consoleFull) for PR 2294 at commit [`272feb9`](https://github.com/apache/spark/commit/272feb9f63517c52e1991988b4fbd8869a992dc4).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-56119499
[QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20562/consoleFull) for PR 2294 at commit [`88814ed`](https://github.com/apache/spark/commit/88814ed215e81bb6b945c8fff60bdb822ac2cb96).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17709099
--- Diff: mllib/src/test/scala/org/apache/spark/mllib/linalg/MatricesSuite.scala ---
@@ -36,4 +36,79 @@ class MatricesSuite extends FunSuite {
Matrices.dense(3, 2, Array(0.0, 1.0, 2.0))
}
}
+
+ test("sparse matrix construction") {
+ val m = 3
+ val n = 2
+ val values = Array(1.0, 2.0, 4.0, 5.0)
+ val colIndices = Array(0, 2, 4)
+ val rowIndices = Array(1, 2, 1, 2)
+ val mat = Matrices.sparse(m, n, colIndices, rowIndices, values).asInstanceOf[SparseMatrix]
+ assert(mat.numRows === m)
+ assert(mat.numCols === n)
+ assert(mat.values.eq(values), "should not copy data")
--- End diff --
also check `colPtrs` and `rowIndices`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17709070
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +201,368 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ if (alpha == 0.0) {
+ logWarning("gemm: alpha is equal to 0. Returning C.")
+ } else {
+ A match {
+ case sparse: SparseMatrix =>
+ gemm(transA, transB, alpha, sparse, B, beta, C)
+ case dense: DenseMatrix =>
+ gemm(transA, transB, alpha, dense, B, beta, C)
+ case _ =>
+ throw new IllegalArgumentException(s"gemm doesn't support matrix type ${A.getClass}.")
+ }
+ }
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ gemm(false, false, alpha, A, B, beta, C)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `DenseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: DenseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+ val tAstr = if (!transA) "N" else "T"
+ val tBstr = if (!transB) "N" else "T"
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ nativeBLAS.dgemm(tAstr, tBstr, mA, nB, kA, alpha, A.values, A.numRows, B.values, B.numRows,
+ beta, C.values, C.numRows)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `SparseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: SparseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ val Avals = A.values
+ val Arows = if (!transA) A.rowIndices else A.colPtrs
+ val Acols = if (!transA) A.colPtrs else A.rowIndices
+
+ // Slicing is easy in this case. This is the optimal multiplication setting for sparse matrices
+ if (transA){
+ var colCounterForB = 0
+ if (!transB){ // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var rowCounterForA = 0
+ val Cstart = colCounterForB * mA
+ val Bstart = colCounterForB * kA
+ while (rowCounterForA < mA) {
+ var i = Arows(rowCounterForA)
+ val indEnd = Arows(rowCounterForA + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B.values(Bstart + Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounterForA
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ } else {
+ while (colCounterForB < nB) {
+ var rowCounter = 0
+ val Cstart = colCounterForB * mA
+ while (rowCounter < mA) {
+ var i = Arows(rowCounter)
+ val indEnd = Arows(rowCounter + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B(colCounterForB, Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounter
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounter += 1
+ }
+ colCounterForB += 1
+ }
+ }
+ } else {
+ // Scale matrix first if `beta` is not equal to 0.0
+ if (beta != 0.0){
+ nativeBLAS.dscal(C.values.length, beta, C.values, 1)
+ }
+ // Perform matrix multiplication and add to C. The rows of A are multiplied by the columns of
+ // B, and added to C.
+ var colCounterForB = 0 // the column to be updated in C
+ if (!transB) { // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var colCounterForA = 0 // The column of A to multiply with the row of B
+ while (colCounterForA < kA){
+ var i = Acols(colCounterForA)
+ val indEnd = Acols(colCounterForA + 1)
+ val Bval = B(colCounterForA, colCounterForB)
+ val Cstart = colCounterForB * mA
+ while (i < indEnd){
+ C.values(Cstart + Arows(i)) += Avals(i) * Bval * alpha
+ i += 1
+ }
+ colCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ } else {
+ while (colCounterForB < nB) {
+ var colCounterForA = 0 // The column of A to multiply with the row of B
+ while (colCounterForA < kA){
+ var i = Acols(colCounterForA)
+ val indEnd = Acols(colCounterForA + 1)
+ val Bval = B(colCounterForB, colCounterForA)
+ val Cstart = colCounterForB * mA
+ while (i < indEnd){
+ C.values(Cstart + Arows(i)) += Avals(i) * Bval * alpha
+ i += 1
+ }
+ colCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ }
+ }
+ }
+
+ /**
+ * y := alpha * A * x + beta * y
+ * @param trans specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ * @param beta a scalar that can be used to scale vector y.
+ * @param y the resulting vector y. Size of m x 1.
+ */
+ def gemv(
+ trans: Boolean,
+ alpha: Double,
+ A: Matrix,
+ x: DenseVector,
+ beta: Double,
+ y: DenseVector): Unit = {
+
+ val mA: Int = if (!trans) A.numRows else A.numCols
+ val nx: Int = x.size
+ val nA: Int = if (!trans) A.numCols else A.numRows
+
+ require(nA == nx, s"The columns of A don't match the number of elements of x. A: $nA, x: $nx")
+ require(mA == y.size,
+ s"The rows of A don't match the number of elements of y. A: $mA, y:${y.size}}")
+ if (alpha == 0.0) {
+ logWarning("gemv: alpha is equal to 0. Returning y.")
+ } else {
+ A match {
+ case sparse: SparseMatrix =>
+ gemv(trans, alpha, sparse, x, beta, y)
+ case dense: DenseMatrix =>
+ gemv(trans, alpha, dense, x, beta, y)
+ case _ =>
+ throw new IllegalArgumentException(s"gemv doesn't support matrix type ${A.getClass}.")
+ }
+ }
+ }
+
+ /**
+ * y := alpha * A * x + beta * y
+ *
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ * @param beta a scalar that can be used to scale vector y.
+ * @param y the resulting vector y. Size of m x 1.
+ */
+ def gemv(
+ alpha: Double,
+ A: Matrix,
+ x: DenseVector,
+ beta: Double,
+ y: DenseVector): Unit = {
+ gemv(false, alpha, A, x, beta, y)
+ }
+
+ /**
+ * y := alpha * A * x
+ *
+ * @param trans specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ *
+ * @return `DenseVector` y, the result of the matrix-vector multiplication. Size of m x 1.
+ */
+ def gemv(
+ trans: Boolean,
+ alpha: Double,
+ A: Matrix,
+ x: DenseVector): DenseVector = {
+ val m = if(!trans) A.numRows else A.numCols
+
+ val y: DenseVector = new DenseVector(Array.fill(m)(0.0))
+ gemv(trans, alpha, A, x, 0.0, y)
+
+ y
+ }
+
+ /**
+ * y := alpha * A * x
+ *
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ *
+ * @return `DenseVector` y, the result of the matrix-vector multiplication. Size of m x 1.
+ */
+ def gemv(
+ alpha: Double,
+ A: Matrix,
+ x: DenseVector): DenseVector = {
+ gemv(false, alpha, A, x)
+ }
+
+
+ /**
+ * y := alpha * A * x + beta * y
+ * For `DenseMatrix` A.
+ */
+ private def gemv(
+ trans: Boolean,
+ alpha: Double,
+ A: DenseMatrix,
+ x: DenseVector,
+ beta: Double,
+ y: DenseVector): Unit = {
+ val tStrA = if (!trans) "N" else "T"
+ nativeBLAS.dgemv(tStrA, A.numRows, A.numCols, alpha, A.values, A.numRows, x.values, 1, beta,
+ y.values, 1)
+ }
+
+ /**
+ * y := alpha * A * x + beta * y
+ * For `SparseMatrix` A.
+ */
+ private def gemv(
+ trans: Boolean,
+ alpha: Double,
+ A: SparseMatrix,
+ x: DenseVector,
+ beta: Double,
+ y: DenseVector): Unit = {
+
+ val mA: Int = if(!trans) A.numRows else A.numCols
+ val nA: Int = if(!trans) A.numCols else A.numRows
+
+ val Avals = A.values
+ val Arows = if (!trans) A.rowIndices else A.colPtrs
+ val Acols = if (!trans) A.colPtrs else A.rowIndices
+
+ // Slicing is easy in this case. This is the optimal multiplication setting for sparse matrices
+ if (trans){
+ var rowCounter = 0
+ while (rowCounter < mA){
+ var i = Arows(rowCounter)
+ var sum = 0.0
+ while(i < Arows(rowCounter + 1)){
+ sum += Avals(i) * x.values(Acols(i))
+ i += 1
+ }
+ y.values(rowCounter) = beta * y.values(rowCounter) + sum * alpha
+ rowCounter += 1
+ }
+ } else {
+ // Scale vector first if `beta` is not equal to 0.0
+ if (beta != 0.0){
+ scal(beta, y)
+ }
+ // Perform matrix-vector multiplication and add to y
+ var colCounterForA = 0
+ while (colCounterForA < nA){
+ var i = Acols(colCounterForA)
+ while (i < Acols(colCounterForA + 1)){
+ val rowIndex = Arows(i)
+ y.values(rowIndex) += Avals(i) * x.values(colCounterForA) * alpha
--- End diff --
cache `x.values(colCounterForA) * alpha`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17704455
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +201,368 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ if (alpha == 0.0) {
+ logWarning("gemm: alpha is equal to 0. Returning C.")
+ } else {
+ A match {
+ case sparse: SparseMatrix =>
+ gemm(transA, transB, alpha, sparse, B, beta, C)
+ case dense: DenseMatrix =>
+ gemm(transA, transB, alpha, dense, B, beta, C)
+ case _ =>
+ throw new IllegalArgumentException(s"gemm doesn't support matrix type ${A.getClass}.")
+ }
+ }
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ gemm(false, false, alpha, A, B, beta, C)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `DenseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: DenseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+ val tAstr = if (!transA) "N" else "T"
+ val tBstr = if (!transB) "N" else "T"
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ nativeBLAS.dgemm(tAstr, tBstr, mA, nB, kA, alpha, A.values, A.numRows, B.values, B.numRows,
+ beta, C.values, C.numRows)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `SparseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: SparseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ val Avals = A.values
+ val Arows = if (!transA) A.rowIndices else A.colPtrs
+ val Acols = if (!transA) A.colPtrs else A.rowIndices
+
+ // Slicing is easy in this case. This is the optimal multiplication setting for sparse matrices
+ if (transA){
+ var colCounterForB = 0
+ if (!transB){ // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var rowCounterForA = 0
+ val Cstart = colCounterForB * mA
+ val Bstart = colCounterForB * kA
+ while (rowCounterForA < mA) {
+ var i = Arows(rowCounterForA)
+ val indEnd = Arows(rowCounterForA + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B.values(Bstart + Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounterForA
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ } else {
+ while (colCounterForB < nB) {
+ var rowCounter = 0
+ val Cstart = colCounterForB * mA
+ while (rowCounter < mA) {
+ var i = Arows(rowCounter)
+ val indEnd = Arows(rowCounter + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B(colCounterForB, Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounter
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounter += 1
+ }
+ colCounterForB += 1
+ }
+ }
+ } else {
+ // Scale matrix first if `beta` is not equal to 0.0
+ if (beta != 0.0){
+ nativeBLAS.dscal(C.values.length, beta, C.values, 1)
+ }
+ // Perform matrix multiplication and add to C. The rows of A are multiplied by the columns of
+ // B, and added to C.
+ var colCounterForB = 0 // the column to be updated in C
+ if (!transB) { // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var colCounterForA = 0 // The column of A to multiply with the row of B
+ while (colCounterForA < kA){
+ var i = Acols(colCounterForA)
+ val indEnd = Acols(colCounterForA + 1)
+ val Bval = B(colCounterForA, colCounterForB)
+ val Cstart = colCounterForB * mA
+ while (i < indEnd){
+ C.values(Cstart + Arows(i)) += Avals(i) * Bval * alpha
+ i += 1
+ }
+ colCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ } else {
+ while (colCounterForB < nB) {
+ var colCounterForA = 0 // The column of A to multiply with the row of B
+ while (colCounterForA < kA){
+ var i = Acols(colCounterForA)
+ val indEnd = Acols(colCounterForA + 1)
+ val Bval = B(colCounterForB, colCounterForA)
+ val Cstart = colCounterForB * mA
+ while (i < indEnd){
+ C.values(Cstart + Arows(i)) += Avals(i) * Bval * alpha
--- End diff --
ditto: cache `Bval * alpha`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17278816
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +199,369 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ A match {
+ case sparse: SparseMatrix =>
+ gemm(transA, transB, alpha, sparse, B, beta, C)
+ case dense: DenseMatrix =>
+ gemm(transA, transB, alpha, dense, B, beta, C)
+ case _ =>
+ throw new IllegalArgumentException(s"gemm doesn't support matrix type ${A.getClass}.")
+ }
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
+ gemm(false, false, alpha, A, B, beta, C)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `DenseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: DenseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+ val tAstr = if (!transA) "N" else "T"
+ val tBstr = if (!transB) "N" else "T"
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ nativeBLAS.dgemm(tAstr, tBstr, mA, nB, kA, alpha, A.values, A.numRows, B.values, B.numRows,
+ beta, C.values, C.numRows)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `SparseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: SparseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ val Avals = A.values
+ val Arows = if (!transA) A.rowIndices else A.colPtrs
+ val Acols = if (!transA) A.colPtrs else A.rowIndices
+
+ // Slicing is easy in this case. This is the optimal multiplication setting for sparse matrices
+ if (transA){
+ var colCounterForB = 0
+ if (!transB){ // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var rowCounterForA = 0
+ val Cstart = colCounterForB * mA
+ while (rowCounterForA < mA) {
+ var i = Arows(rowCounterForA)
+ val indEnd = Arows(rowCounterForA + 1)
+ val Bstart = colCounterForB * kA
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B(Bstart + Acols(i))
--- End diff --
`B` -> `B.values`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-54685631
[QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19810/consoleFull) for PR 2294 at commit [`4362ff1`](https://github.com/apache/spark/commit/4362ff1d6b79604b9919da1d550ed466021b8f5c).
* This patch **fails** unit tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `class SparseMatrix(val numRows: Int,`
* `case class AddJar(path: String) extends LeafNode with Command `
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17255528
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala ---
@@ -63,7 +88,162 @@ class DenseMatrix(val numRows: Int, val numCols: Int, val values: Array[Double])
override def toArray: Array[Double] = values
- private[mllib] override def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+ private [mllib] def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+
+ private [mllib] def apply(i: Int): Double = values(i)
+
+ private [mllib] def apply(r: Int, c: Int): Double = values(index(r, c))
+
+ private [mllib] def index(r: Int, c: Int): Int = r + numRows * c
+
+ private [mllib] def update(r: Int, c: Int, v: Double){
+ values(index(r, c)) = v
+ }
+
+ def copy = new DenseMatrix(numRows, numCols, values.clone())
+}
+
+/**
+ * Factory methods for [[org.apache.spark.mllib.linalg.DenseMatrix]].
+ *
+ * These methods can be used to generate common matrix types such as the Identity matrix, any
+ * diagonal matrix, zero matrix, and random matrices.
+ */
+object DenseMatrix {
+
+ /**
+ * Generate a `DenseMatrix` consisting of zeros.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values of zeros
+ */
+ def zeros(numRows: Int, numCols: Int) =
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(0.0))
+
+ /**
+ * Generate a `DenseMatrix` consisting of ones.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values of ones
+ */
+ def ones(numRows: Int, numCols: Int) =
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(1.0))
+
+ /**
+ * Generate an Identity Matrix in `DenseMatrix` format.
+ * @param n number of rows and columns of the matrix
+ * @return `DenseMatrix` with size `n` x `n` and values of ones on the diagonal
+ */
+ def eye(n: Int) = {
+ val identity = DenseMatrix.zeros(n,n)
+ for (i <- 0 until n){
+ identity.update(i, i, 1.0)
+ }
+ identity
+ }
+
+ /**
+ * Generate a `DenseMatrix` consisting of i.i.d. uniform random numbers.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values in U(0, 1)
+ */
+ def rand(numRows: Int, numCols: Int) = {
+ val rand = new scala.util.Random
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(rand.nextDouble()))
+ }
+
+ /**
+ * Generate a `DenseMatrix` consisting of i.i.d. gaussian random numbers.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values in N(0, 1)
+ */
+ def randn(numRows: Int, numCols: Int) = {
+ val rand = new scala.util.Random
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(rand.nextGaussian()))
+ }
+
+ /**
+ * Generate a diagonal matrix in `DenseMatrix` format from the supplied values.
+ * @param values values on the diagonal of the matrix
+ * @return Square `DenseMatrix` with size `values.length` x `values.length` and `values`
+ * on the diagonal
+ */
+ def diag(values: Array[Double]) = {
+ val n = values.length
+ val matrix = DenseMatrix.eye(n)
+ for (i <- 0 until n) matrix.update(i, i, values(i))
+ matrix
+ }
+}
+
+/**
+ * Column-majored sparse matrix.
+ * The entry values are stored in Compressed-Column Storage (CCS) format.
+ * For example, the following matrix
+ * {{{
+ * 1.0 0.0 4.0
+ * 0.0 3.0 5.0
+ * 2.0 0.0 6.0
+ * }}}
+ * is stored as `values: [1.0, 2.0, 3.0, 4.0, 5.0, 6.0]`,
+ * `rowIndices=[0, 2, 1, 0, 1, 2]`, `colPointers=[0, 2, 3, 6]`.
+ *
+ * @param numRows number of rows
+ * @param numCols number of columns
+ * @param colPointers the index corresponding to the start of a new column
+ * @param rowIndices the row index of the entry
+ * @param values non-zero matrix entries in column major
+ */
+class SparseMatrix(
+ val numRows: Int,
+ val numCols: Int,
+ val colPointers: Array[Int],
--- End diff --
We can use either `columnPointers` or `colPtrs`. The latter seems better to me.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17255357
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala ---
@@ -63,7 +88,162 @@ class DenseMatrix(val numRows: Int, val numCols: Int, val values: Array[Double])
override def toArray: Array[Double] = values
- private[mllib] override def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+ private [mllib] def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+
+ private [mllib] def apply(i: Int): Double = values(i)
+
+ private [mllib] def apply(r: Int, c: Int): Double = values(index(r, c))
+
+ private [mllib] def index(r: Int, c: Int): Int = r + numRows * c
+
+ private [mllib] def update(r: Int, c: Int, v: Double){
+ values(index(r, c)) = v
+ }
+
+ def copy = new DenseMatrix(numRows, numCols, values.clone())
+}
+
+/**
+ * Factory methods for [[org.apache.spark.mllib.linalg.DenseMatrix]].
+ *
+ * These methods can be used to generate common matrix types such as the Identity matrix, any
+ * diagonal matrix, zero matrix, and random matrices.
+ */
+object DenseMatrix {
+
+ /**
+ * Generate a `DenseMatrix` consisting of zeros.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values of zeros
+ */
+ def zeros(numRows: Int, numCols: Int) =
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(0.0))
+
+ /**
+ * Generate a `DenseMatrix` consisting of ones.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values of ones
+ */
+ def ones(numRows: Int, numCols: Int) =
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(1.0))
+
+ /**
+ * Generate an Identity Matrix in `DenseMatrix` format.
+ * @param n number of rows and columns of the matrix
+ * @return `DenseMatrix` with size `n` x `n` and values of ones on the diagonal
+ */
+ def eye(n: Int) = {
+ val identity = DenseMatrix.zeros(n,n)
+ for (i <- 0 until n){
+ identity.update(i, i, 1.0)
+ }
+ identity
+ }
+
+ /**
+ * Generate a `DenseMatrix` consisting of i.i.d. uniform random numbers.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values in U(0, 1)
+ */
+ def rand(numRows: Int, numCols: Int) = {
+ val rand = new scala.util.Random
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(rand.nextDouble()))
+ }
+
+ /**
+ * Generate a `DenseMatrix` consisting of i.i.d. gaussian random numbers.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values in N(0, 1)
+ */
+ def randn(numRows: Int, numCols: Int) = {
+ val rand = new scala.util.Random
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(rand.nextGaussian()))
+ }
+
+ /**
+ * Generate a diagonal matrix in `DenseMatrix` format from the supplied values.
+ * @param values values on the diagonal of the matrix
+ * @return Square `DenseMatrix` with size `values.length` x `values.length` and `values`
+ * on the diagonal
+ */
+ def diag(values: Array[Double]) = {
+ val n = values.length
+ val matrix = DenseMatrix.eye(n)
+ for (i <- 0 until n) matrix.update(i, i, values(i))
+ matrix
+ }
+}
+
+/**
+ * Column-majored sparse matrix.
+ * The entry values are stored in Compressed-Column Storage (CCS) format.
+ * For example, the following matrix
+ * {{{
+ * 1.0 0.0 4.0
+ * 0.0 3.0 5.0
+ * 2.0 0.0 6.0
+ * }}}
+ * is stored as `values: [1.0, 2.0, 3.0, 4.0, 5.0, 6.0]`,
+ * `rowIndices=[0, 2, 1, 0, 1, 2]`, `colPointers=[0, 2, 3, 6]`.
+ *
+ * @param numRows number of rows
+ * @param numCols number of columns
+ * @param colPointers the index corresponding to the start of a new column
+ * @param rowIndices the row index of the entry
+ * @param values non-zero matrix entries in column major
+ */
+class SparseMatrix(
+ val numRows: Int,
+ val numCols: Int,
+ val colPointers: Array[Int],
+ val rowIndices: Array[Int],
+ val values: Array[Double]) extends Matrix {
+
+ require(values.length == rowIndices.length, "The number of row indices and values don't match!")
+ require(colPointers.length == numCols + 1, "The length of the column indices should be the " +
+ s"number of columns + 1. Currently, colPointers.length: ${colPointers.length}, " +
+ s"numCols: $numCols")
+
+ override def toArray: Array[Double] = values
+
+ private [mllib] def toBreeze: BM[Double] =
+ new BSM[Double](values, numRows, numCols, colPointers, rowIndices)
+
+ private [mllib] def apply(i: Int): Double = values(i)
+
+ private [mllib] def apply(r: Int, c: Int): Double = {
+ val ind = index(r,c)
+ if (ind == -1) 0.0 else values(ind)
+ }
+
+ private [mllib] def index(r: Int, c: Int): Int = {
+ val regionStart = colPointers(c)
+ val regionEnd = colPointers(c + 1)
+ val region = rowIndices.slice(regionStart, regionEnd)
+ if (region.contains(r)){
+ region.indexOf(r) + regionStart
+ } else {
+ -1 // return zero element
+ }
+ }
+
+ // TODO(Burak): Maybe convert to Breeze to update zero entries? I can't think of any MLlib
+ // TODO: algorithm that would use mutable Sparse Matrices
+ private [mllib] def update(r: Int, c: Int, v: Double){
+ val ind = index(r, c)
+ if (ind == -1){
+ throw new IllegalArgumentException("The given row and column indices correspond to a zero " +
+ "value. Sparse Matrices are currently immutable.")
+ } else {
+ values(index(r, c)) = v
+ }
+ }
+
+ def copy = new SparseMatrix(numRows, numCols, colPointers, rowIndices, values.clone())
--- End diff --
override?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17709086
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala ---
@@ -59,11 +93,113 @@ trait Matrix extends Serializable {
*/
class DenseMatrix(val numRows: Int, val numCols: Int, val values: Array[Double]) extends Matrix {
- require(values.length == numRows * numCols)
+ require(values.length == numRows * numCols, "The number of values supplied doesn't match the " +
+ s"size of the matrix! values.length: ${values.length}, numRows * numCols: ${numRows * numCols}")
override def toArray: Array[Double] = values
- private[mllib] override def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+ private[mllib] def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+
+ private[mllib] def apply(i: Int): Double = values(i)
+
+ private[mllib] def apply(i: Int, j: Int): Double = values(index(i, j))
+
+ private[mllib] def index(i: Int, j: Int): Int = i + numRows * j
+
+ private[mllib] def update(i: Int, j: Int, v: Double){
+ values(index(i, j)) = v
+ }
+
+ override def copy = new DenseMatrix(numRows, numCols, values.clone())
+}
+
+/**
+ * Column-majored sparse matrix.
+ * The entry values are stored in Compressed Sparse Column (CSC) format.
+ * For example, the following matrix
+ * {{{
+ * 1.0 0.0 4.0
+ * 0.0 3.0 5.0
+ * 2.0 0.0 6.0
+ * }}}
+ * is stored as `values: [1.0, 2.0, 3.0, 4.0, 5.0, 6.0]`,
+ * `rowIndices=[0, 2, 1, 0, 1, 2]`, `colPointers=[0, 2, 3, 6]`.
+ *
+ * @param numRows number of rows
+ * @param numCols number of columns
+ * @param colPtrs the index corresponding to the start of a new column
+ * @param rowIndices the row index of the entry
+ * @param values non-zero matrix entries in column major
+ */
+class SparseMatrix(
+ val numRows: Int,
+ val numCols: Int,
+ val colPtrs: Array[Int],
+ val rowIndices: Array[Int],
+ val values: Array[Double]) extends Matrix {
+
+ require(values.length == rowIndices.length, "The number of row indices and values don't match! " +
+ s"values.length: ${values.length}, rowIndices.length: ${rowIndices.length}")
+ require(colPtrs.length == numCols + 1, "The length of the column indices should be the " +
+ s"number of columns + 1. Currently, colPointers.length: ${colPtrs.length}, " +
+ s"numCols: $numCols")
+
+ override def toArray: Array[Double] = {
+ val arr = Array.fill(numRows * numCols)(0.0)
+ var j = 0
+ while (j < numCols) {
+ var i = colPtrs(j)
+ val indEnd = colPtrs(j + 1)
+ val offset = j * numRows
+ while (i < indEnd) {
+ val rowIndex = rowIndices(i)
+ arr(offset + rowIndex) = values(i)
+ i += 1
+ }
+ j += 1
+ }
+ arr
+ }
+
+ private[mllib] def toBreeze: BM[Double] =
+ new BSM[Double](values, numRows, numCols, colPtrs, rowIndices)
+
+ private[mllib] def apply(i: Int): Double = values(i)
+
+ private[mllib] def apply(i: Int, j: Int): Double = {
+ val ind = index(i, j)
+ if (ind == -1) 0.0 else values(ind)
+ }
+
+ private[mllib] def index(i: Int, j: Int): Int = {
+ var regionStart = colPtrs(j)
--- End diff --
use `Arrays.binarySearch`: http://docs.oracle.com/javase/7/docs/api/java/util/Arrays.html#binarySearch(int[],%20int,%20int,%20int)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17709097
--- Diff: mllib/src/test/scala/org/apache/spark/mllib/linalg/MatricesSuite.scala ---
@@ -36,4 +36,79 @@ class MatricesSuite extends FunSuite {
Matrices.dense(3, 2, Array(0.0, 1.0, 2.0))
}
}
+
+ test("sparse matrix construction") {
+ val m = 3
+ val n = 2
+ val values = Array(1.0, 2.0, 4.0, 5.0)
+ val colIndices = Array(0, 2, 4)
--- End diff --
`colPtrs`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-54849222
test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-54694440
[QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19815/consoleFull) for PR 2294 at commit [`8dcb763`](https://github.com/apache/spark/commit/8dcb7632093fc2d87a626b85e08ddabcbf119543).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-54859801
[QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19982/consoleFull) for PR 2294 at commit [`848406c`](https://github.com/apache/spark/commit/848406c1b6779eea9bbcf1dd582b541db46d7dad).
* This patch **fails** unit tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `class SparseMatrix(`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-56094356
[QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20548/consoleFull) for PR 2294 at commit [`d162684`](https://github.com/apache/spark/commit/d16268496cefe06d8818b20721eede56a9de41a2).
* This patch **fails** unit tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `sealed trait Matrix extends Serializable `
* `class SparseMatrix(`
* `sealed trait Vector extends Serializable `
* ` Seq( // Ignore new functions added to trait `Matrix`. They are sealed traits now.`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/2294
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-56107408
[QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20550/consoleFull) for PR 2294 at commit [`272feb9`](https://github.com/apache/spark/commit/272feb9f63517c52e1991988b4fbd8869a992dc4).
* This patch **fails** unit tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `sealed trait Matrix extends Serializable `
* `class SparseMatrix(`
* `sealed trait Vector extends Serializable `
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17709101
--- Diff: mllib/src/test/scala/org/apache/spark/mllib/linalg/MatricesSuite.scala ---
@@ -36,4 +36,79 @@ class MatricesSuite extends FunSuite {
Matrices.dense(3, 2, Array(0.0, 1.0, 2.0))
}
}
+
+ test("sparse matrix construction") {
+ val m = 3
+ val n = 2
+ val values = Array(1.0, 2.0, 4.0, 5.0)
+ val colIndices = Array(0, 2, 4)
+ val rowIndices = Array(1, 2, 1, 2)
+ val mat = Matrices.sparse(m, n, colIndices, rowIndices, values).asInstanceOf[SparseMatrix]
+ assert(mat.numRows === m)
+ assert(mat.numCols === n)
+ assert(mat.values.eq(values), "should not copy data")
+ }
+
+ test("sparse matrix construction with wrong number of elements") {
+ intercept[RuntimeException] {
--- End diff --
It should be `IllegalArgumentException` (thrown by `require`)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-55828361
[QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20432/consoleFull) for PR 2294 at commit [`d510c8f`](https://github.com/apache/spark/commit/d510c8f940faee3bdb2b00306f150bc99630396b).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17709058
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +201,368 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ if (alpha == 0.0) {
+ logWarning("gemm: alpha is equal to 0. Returning C.")
+ } else {
+ A match {
+ case sparse: SparseMatrix =>
+ gemm(transA, transB, alpha, sparse, B, beta, C)
+ case dense: DenseMatrix =>
+ gemm(transA, transB, alpha, dense, B, beta, C)
+ case _ =>
+ throw new IllegalArgumentException(s"gemm doesn't support matrix type ${A.getClass}.")
+ }
+ }
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ gemm(false, false, alpha, A, B, beta, C)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `DenseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: DenseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+ val tAstr = if (!transA) "N" else "T"
+ val tBstr = if (!transB) "N" else "T"
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ nativeBLAS.dgemm(tAstr, tBstr, mA, nB, kA, alpha, A.values, A.numRows, B.values, B.numRows,
+ beta, C.values, C.numRows)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `SparseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: SparseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ val Avals = A.values
+ val Arows = if (!transA) A.rowIndices else A.colPtrs
+ val Acols = if (!transA) A.colPtrs else A.rowIndices
+
+ // Slicing is easy in this case. This is the optimal multiplication setting for sparse matrices
+ if (transA){
+ var colCounterForB = 0
+ if (!transB){ // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var rowCounterForA = 0
+ val Cstart = colCounterForB * mA
+ val Bstart = colCounterForB * kA
+ while (rowCounterForA < mA) {
+ var i = Arows(rowCounterForA)
+ val indEnd = Arows(rowCounterForA + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B.values(Bstart + Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounterForA
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ } else {
+ while (colCounterForB < nB) {
+ var rowCounter = 0
+ val Cstart = colCounterForB * mA
+ while (rowCounter < mA) {
+ var i = Arows(rowCounter)
+ val indEnd = Arows(rowCounter + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B(colCounterForB, Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounter
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounter += 1
+ }
+ colCounterForB += 1
+ }
+ }
+ } else {
+ // Scale matrix first if `beta` is not equal to 0.0
+ if (beta != 0.0){
+ nativeBLAS.dscal(C.values.length, beta, C.values, 1)
+ }
+ // Perform matrix multiplication and add to C. The rows of A are multiplied by the columns of
+ // B, and added to C.
+ var colCounterForB = 0 // the column to be updated in C
+ if (!transB) { // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var colCounterForA = 0 // The column of A to multiply with the row of B
+ while (colCounterForA < kA){
+ var i = Acols(colCounterForA)
+ val indEnd = Acols(colCounterForA + 1)
+ val Bval = B(colCounterForA, colCounterForB)
+ val Cstart = colCounterForB * mA
+ while (i < indEnd){
+ C.values(Cstart + Arows(i)) += Avals(i) * Bval * alpha
+ i += 1
+ }
+ colCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ } else {
+ while (colCounterForB < nB) {
+ var colCounterForA = 0 // The column of A to multiply with the row of B
+ while (colCounterForA < kA){
+ var i = Acols(colCounterForA)
+ val indEnd = Acols(colCounterForA + 1)
+ val Bval = B(colCounterForB, colCounterForA)
+ val Cstart = colCounterForB * mA
+ while (i < indEnd){
+ C.values(Cstart + Arows(i)) += Avals(i) * Bval * alpha
+ i += 1
+ }
+ colCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ }
+ }
+ }
+
+ /**
+ * y := alpha * A * x + beta * y
+ * @param trans specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ * @param beta a scalar that can be used to scale vector y.
+ * @param y the resulting vector y. Size of m x 1.
+ */
+ def gemv(
+ trans: Boolean,
+ alpha: Double,
+ A: Matrix,
+ x: DenseVector,
+ beta: Double,
+ y: DenseVector): Unit = {
+
+ val mA: Int = if (!trans) A.numRows else A.numCols
+ val nx: Int = x.size
+ val nA: Int = if (!trans) A.numCols else A.numRows
+
+ require(nA == nx, s"The columns of A don't match the number of elements of x. A: $nA, x: $nx")
+ require(mA == y.size,
+ s"The rows of A don't match the number of elements of y. A: $mA, y:${y.size}}")
+ if (alpha == 0.0) {
+ logWarning("gemv: alpha is equal to 0. Returning y.")
--- End diff --
ditto: `logDebug`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by brkyvz <gi...@git.apache.org>.
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-55833607
@mengxr, do you know what this mima issue is by any chance? What do I have to fix?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17255313
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala ---
@@ -63,7 +88,162 @@ class DenseMatrix(val numRows: Int, val numCols: Int, val values: Array[Double])
override def toArray: Array[Double] = values
- private[mllib] override def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+ private [mllib] def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+
+ private [mllib] def apply(i: Int): Double = values(i)
+
+ private [mllib] def apply(r: Int, c: Int): Double = values(index(r, c))
+
+ private [mllib] def index(r: Int, c: Int): Int = r + numRows * c
+
+ private [mllib] def update(r: Int, c: Int, v: Double){
+ values(index(r, c)) = v
+ }
+
+ def copy = new DenseMatrix(numRows, numCols, values.clone())
+}
+
+/**
+ * Factory methods for [[org.apache.spark.mllib.linalg.DenseMatrix]].
+ *
+ * These methods can be used to generate common matrix types such as the Identity matrix, any
+ * diagonal matrix, zero matrix, and random matrices.
+ */
+object DenseMatrix {
+
+ /**
+ * Generate a `DenseMatrix` consisting of zeros.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values of zeros
+ */
+ def zeros(numRows: Int, numCols: Int) =
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(0.0))
+
+ /**
+ * Generate a `DenseMatrix` consisting of ones.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values of ones
+ */
+ def ones(numRows: Int, numCols: Int) =
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(1.0))
+
+ /**
+ * Generate an Identity Matrix in `DenseMatrix` format.
+ * @param n number of rows and columns of the matrix
+ * @return `DenseMatrix` with size `n` x `n` and values of ones on the diagonal
+ */
+ def eye(n: Int) = {
+ val identity = DenseMatrix.zeros(n,n)
+ for (i <- 0 until n){
--- End diff --
Use `while`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17255363
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala ---
@@ -83,16 +263,36 @@ object Matrices {
}
/**
+ * Creates a column-majored sparse matrix in Compressed Column Storage (CCS) format.
+ *
+ * @param numRows number of rows
+ * @param numCols number of columns
+ * @param colPointers the index corresponding to the start of a new column
+ * @param rowIndices the row index of the entry
+ * @param values non-zero matrix entries in column major
+ */
+ def sparse(
+ numRows: Int,
+ numCols: Int,
+ colPointers: Array[Int],
+ rowIndices: Array[Int],
+ values: Array[Double]): Matrix = {
+ new SparseMatrix(numRows, numCols, colPointers, rowIndices, values)
+ }
+
+ /**
* Creates a Matrix instance from a breeze matrix.
* @param breeze a breeze matrix
* @return a Matrix instance
*/
- private[mllib] def fromBreeze(breeze: BM[Double]): Matrix = {
+ private [mllib] def fromBreeze(breeze: BM[Double]): Matrix = {
--- End diff --
remove the space
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-55389216
[QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20225/consoleFull) for PR 2294 at commit [`a85ccb7`](https://github.com/apache/spark/commit/a85ccb712d83d20b178b40c32bd473d7d018a88f).
* This patch **fails** unit tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `class SparseMatrix(`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17704445
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +201,368 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ if (alpha == 0.0) {
+ logWarning("gemm: alpha is equal to 0. Returning C.")
--- End diff --
`gemm` maybe called many times and there are cases when `alpha` is indeed zero. Maybe we can change it to `logDebug` to avoid outputting many log messages at a normal level.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17278854
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +199,369 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ A match {
+ case sparse: SparseMatrix =>
+ gemm(transA, transB, alpha, sparse, B, beta, C)
+ case dense: DenseMatrix =>
+ gemm(transA, transB, alpha, dense, B, beta, C)
+ case _ =>
+ throw new IllegalArgumentException(s"gemm doesn't support matrix type ${A.getClass}.")
+ }
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
+ gemm(false, false, alpha, A, B, beta, C)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `DenseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: DenseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+ val tAstr = if (!transA) "N" else "T"
+ val tBstr = if (!transB) "N" else "T"
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ nativeBLAS.dgemm(tAstr, tBstr, mA, nB, kA, alpha, A.values, A.numRows, B.values, B.numRows,
+ beta, C.values, C.numRows)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `SparseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: SparseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ val Avals = A.values
+ val Arows = if (!transA) A.rowIndices else A.colPtrs
+ val Acols = if (!transA) A.colPtrs else A.rowIndices
+
+ // Slicing is easy in this case. This is the optimal multiplication setting for sparse matrices
+ if (transA){
+ var colCounterForB = 0
+ if (!transB){ // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var rowCounterForA = 0
+ val Cstart = colCounterForB * mA
+ while (rowCounterForA < mA) {
+ var i = Arows(rowCounterForA)
+ val indEnd = Arows(rowCounterForA + 1)
+ val Bstart = colCounterForB * kA
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B(Bstart + Acols(i))
+ i += 1
+ }
+ C.values(rowCounterForA + Cstart) = beta * C.values(rowCounterForA + Cstart) + sum
+ rowCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ } else {
+ while (colCounterForB < nB) {
+ var rowCounter = 0
+ val Cstart = colCounterForB * mA
+ while (rowCounter < mA) {
+ var i = Arows(rowCounter)
+ val indEnd = Arows(rowCounter + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B(colCounterForB, Acols(i))
+ i += 1
+ }
+ C.values(rowCounter + Cstart) = beta * C.values(rowCounter + Cstart) + sum
+ rowCounter += 1
+ }
+ colCounterForB += 1
+ }
+ }
+ } else {
+ // Scale matrix first if `beta` is not equal to 0.0
+ if (beta != 0.0){
+ var i = 0
--- End diff --
use `dscal` on `C.values`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17709069
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +201,368 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ if (alpha == 0.0) {
+ logWarning("gemm: alpha is equal to 0. Returning C.")
+ } else {
+ A match {
+ case sparse: SparseMatrix =>
+ gemm(transA, transB, alpha, sparse, B, beta, C)
+ case dense: DenseMatrix =>
+ gemm(transA, transB, alpha, dense, B, beta, C)
+ case _ =>
+ throw new IllegalArgumentException(s"gemm doesn't support matrix type ${A.getClass}.")
+ }
+ }
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ gemm(false, false, alpha, A, B, beta, C)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `DenseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: DenseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+ val tAstr = if (!transA) "N" else "T"
+ val tBstr = if (!transB) "N" else "T"
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ nativeBLAS.dgemm(tAstr, tBstr, mA, nB, kA, alpha, A.values, A.numRows, B.values, B.numRows,
+ beta, C.values, C.numRows)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `SparseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: SparseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ val Avals = A.values
+ val Arows = if (!transA) A.rowIndices else A.colPtrs
+ val Acols = if (!transA) A.colPtrs else A.rowIndices
+
+ // Slicing is easy in this case. This is the optimal multiplication setting for sparse matrices
+ if (transA){
+ var colCounterForB = 0
+ if (!transB){ // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var rowCounterForA = 0
+ val Cstart = colCounterForB * mA
+ val Bstart = colCounterForB * kA
+ while (rowCounterForA < mA) {
+ var i = Arows(rowCounterForA)
+ val indEnd = Arows(rowCounterForA + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B.values(Bstart + Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounterForA
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ } else {
+ while (colCounterForB < nB) {
+ var rowCounter = 0
+ val Cstart = colCounterForB * mA
+ while (rowCounter < mA) {
+ var i = Arows(rowCounter)
+ val indEnd = Arows(rowCounter + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B(colCounterForB, Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounter
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounter += 1
+ }
+ colCounterForB += 1
+ }
+ }
+ } else {
+ // Scale matrix first if `beta` is not equal to 0.0
+ if (beta != 0.0){
+ nativeBLAS.dscal(C.values.length, beta, C.values, 1)
+ }
+ // Perform matrix multiplication and add to C. The rows of A are multiplied by the columns of
+ // B, and added to C.
+ var colCounterForB = 0 // the column to be updated in C
+ if (!transB) { // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var colCounterForA = 0 // The column of A to multiply with the row of B
+ while (colCounterForA < kA){
+ var i = Acols(colCounterForA)
+ val indEnd = Acols(colCounterForA + 1)
+ val Bval = B(colCounterForA, colCounterForB)
+ val Cstart = colCounterForB * mA
+ while (i < indEnd){
+ C.values(Cstart + Arows(i)) += Avals(i) * Bval * alpha
+ i += 1
+ }
+ colCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ } else {
+ while (colCounterForB < nB) {
+ var colCounterForA = 0 // The column of A to multiply with the row of B
+ while (colCounterForA < kA){
+ var i = Acols(colCounterForA)
+ val indEnd = Acols(colCounterForA + 1)
+ val Bval = B(colCounterForB, colCounterForA)
+ val Cstart = colCounterForB * mA
+ while (i < indEnd){
+ C.values(Cstart + Arows(i)) += Avals(i) * Bval * alpha
+ i += 1
+ }
+ colCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ }
+ }
+ }
+
+ /**
+ * y := alpha * A * x + beta * y
+ * @param trans specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ * @param beta a scalar that can be used to scale vector y.
+ * @param y the resulting vector y. Size of m x 1.
+ */
+ def gemv(
+ trans: Boolean,
+ alpha: Double,
+ A: Matrix,
+ x: DenseVector,
+ beta: Double,
+ y: DenseVector): Unit = {
+
+ val mA: Int = if (!trans) A.numRows else A.numCols
+ val nx: Int = x.size
+ val nA: Int = if (!trans) A.numCols else A.numRows
+
+ require(nA == nx, s"The columns of A don't match the number of elements of x. A: $nA, x: $nx")
+ require(mA == y.size,
+ s"The rows of A don't match the number of elements of y. A: $mA, y:${y.size}}")
+ if (alpha == 0.0) {
+ logWarning("gemv: alpha is equal to 0. Returning y.")
+ } else {
+ A match {
+ case sparse: SparseMatrix =>
+ gemv(trans, alpha, sparse, x, beta, y)
+ case dense: DenseMatrix =>
+ gemv(trans, alpha, dense, x, beta, y)
+ case _ =>
+ throw new IllegalArgumentException(s"gemv doesn't support matrix type ${A.getClass}.")
+ }
+ }
+ }
+
+ /**
+ * y := alpha * A * x + beta * y
+ *
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ * @param beta a scalar that can be used to scale vector y.
+ * @param y the resulting vector y. Size of m x 1.
+ */
+ def gemv(
+ alpha: Double,
+ A: Matrix,
+ x: DenseVector,
+ beta: Double,
+ y: DenseVector): Unit = {
+ gemv(false, alpha, A, x, beta, y)
+ }
+
+ /**
+ * y := alpha * A * x
+ *
+ * @param trans specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ *
+ * @return `DenseVector` y, the result of the matrix-vector multiplication. Size of m x 1.
+ */
+ def gemv(
+ trans: Boolean,
+ alpha: Double,
+ A: Matrix,
+ x: DenseVector): DenseVector = {
+ val m = if(!trans) A.numRows else A.numCols
+
+ val y: DenseVector = new DenseVector(Array.fill(m)(0.0))
+ gemv(trans, alpha, A, x, 0.0, y)
+
+ y
+ }
+
+ /**
+ * y := alpha * A * x
+ *
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ *
+ * @return `DenseVector` y, the result of the matrix-vector multiplication. Size of m x 1.
+ */
+ def gemv(
+ alpha: Double,
+ A: Matrix,
+ x: DenseVector): DenseVector = {
+ gemv(false, alpha, A, x)
+ }
+
+
+ /**
+ * y := alpha * A * x + beta * y
+ * For `DenseMatrix` A.
+ */
+ private def gemv(
+ trans: Boolean,
+ alpha: Double,
+ A: DenseMatrix,
+ x: DenseVector,
+ beta: Double,
+ y: DenseVector): Unit = {
+ val tStrA = if (!trans) "N" else "T"
+ nativeBLAS.dgemv(tStrA, A.numRows, A.numCols, alpha, A.values, A.numRows, x.values, 1, beta,
+ y.values, 1)
+ }
+
+ /**
+ * y := alpha * A * x + beta * y
+ * For `SparseMatrix` A.
+ */
+ private def gemv(
+ trans: Boolean,
+ alpha: Double,
+ A: SparseMatrix,
+ x: DenseVector,
+ beta: Double,
+ y: DenseVector): Unit = {
+
+ val mA: Int = if(!trans) A.numRows else A.numCols
+ val nA: Int = if(!trans) A.numCols else A.numRows
+
+ val Avals = A.values
+ val Arows = if (!trans) A.rowIndices else A.colPtrs
+ val Acols = if (!trans) A.colPtrs else A.rowIndices
+
+ // Slicing is easy in this case. This is the optimal multiplication setting for sparse matrices
+ if (trans){
+ var rowCounter = 0
+ while (rowCounter < mA){
+ var i = Arows(rowCounter)
+ var sum = 0.0
+ while(i < Arows(rowCounter + 1)){
+ sum += Avals(i) * x.values(Acols(i))
+ i += 1
+ }
+ y.values(rowCounter) = beta * y.values(rowCounter) + sum * alpha
+ rowCounter += 1
+ }
+ } else {
+ // Scale vector first if `beta` is not equal to 0.0
+ if (beta != 0.0){
+ scal(beta, y)
+ }
+ // Perform matrix-vector multiplication and add to y
+ var colCounterForA = 0
+ while (colCounterForA < nA){
+ var i = Acols(colCounterForA)
+ while (i < Acols(colCounterForA + 1)){
--- End diff --
ditto: cache the end index
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17255281
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +199,451 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: String,
+ transB: String,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
+ var mA: Int = A.numRows
+ var nB: Int = B.numCols
+ var kA: Int = A.numCols
+ var kB: Int = B.numRows
+
+ if (transA == "T" || transA=="t"){
+ mA = A.numCols
+ kA = A.numRows
+ }
+ require(transA == "T" || transA == "t" || transA == "N" || transA == "n",
+ s"""Invalid argument used for transA: $transA. Must be \"N\", \"n\", \"T\", or \"t\"""")
+ if (transB == "T" || transB=="t"){
+ nB = B.numRows
+ kB = B.numCols
+ }
+ require(transB == "T" || transB == "t" || transB == "N" || transB == "n",
+ s"""Invalid argument used for transB: $transB. Must be \"N\", \"n\", \"T\", or \"t\"""")
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ A match {
+ case sparse: SparseMatrix =>
+ gemm(transA, transB, alpha, sparse, B, beta, C)
+ case dense: DenseMatrix =>
+ gemm(transA, transB, alpha, dense, B, beta, C)
+ case _ =>
+ throw new IllegalArgumentException(s"gemm doesn't support matrix type ${A.getClass}.")
+ }
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
+ gemm("N", "N", alpha, A, B, beta, C)
+ }
+
+ /**
+ * C := alpha * A * B
+ *
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ *
+ * @return The resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: String,
+ transB: String,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix) : DenseMatrix = {
+ var mA: Int = A.numRows
+ var nB: Int = B.numCols
+ var kA: Int = A.numCols
+ var kB: Int = B.numRows
+
+ if (transA == "T" || transA=="T"){
+ mA = A.numCols
+ kA = A.numRows
+ }
+ if (transB == "T" || transB=="T"){
+ nB = B.numRows
+ kB = B.numCols
+ }
+
+ val C: DenseMatrix = DenseMatrix.zeros(mA, nB)
+ gemm(transA, transB, alpha, A, B, 0.0, C)
+
+ C
+ }
+
+ /**
+ * C := alpha * A * B
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ *
+ * @return The resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix) : DenseMatrix = {
+ gemm("N", "N", alpha, A, B)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `DenseMatrix` A.
+ */
+ private def gemm(
+ transA: String,
+ transB: String,
+ alpha: Double,
+ A: DenseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
+ var mA: Int = A.numRows
+ var nB: Int = B.numCols
+ var kA: Int = A.numCols
+ var kB: Int = B.numRows
+
+ if (transA == "T" || transA=="t"){
+ mA = A.numCols
+ kA = A.numRows
+ }
+ if (transB == "T" || transB=="t"){
+ nB = B.numRows
+ kB = B.numCols
+ }
+
+ nativeBLAS.dgemm(transA,transB, mA, nB, kA, alpha, A.toArray, A.numRows, B.toArray, B.numRows,
+ beta, C.toArray, C.numRows)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `SparseMatrix` A.
+ */
+ private def gemm(
+ transA: String,
+ transB: String,
+ alpha: Double,
+ A: SparseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
+ var transposeA = false
+ var transposeB = false
+ var mA: Int = A.numRows
+ var nB: Int = B.numCols
+ var kA: Int = A.numCols
+
+ if (transA == "T" || transA=="t"){
+ mA = A.numCols
+ kA = A.numRows
+ transposeA = true
+ }
+ if (transB == "T" || transB=="t"){
+ nB = B.numRows
+ transposeB = true
+ }
+ val Avals = A.toArray
+ val Arows = if (!transposeA) A.rowIndices else A.colPointers
+ val Acols = if (!transposeA) A.colPointers else A.rowIndices
+
+ // Slicing is easy in this case. This is the optimal multiplication setting for sparse matrices
+ if (transposeA){
+ var colCounter = 0
+ while (colCounter < nB){ // Tests showed that the outer loop being columns was faster
+ var rowCounter = 0
+ while (rowCounter < mA){
+ val indStart = Arows(rowCounter)
+ val indEnd = Arows(rowCounter + 1)
+ var elementCount = 0 // Loop over non-zero entries in column (actually the row indices,
+ // since this is a transposed multiplication
+ var sum = 0.0
+ while(indStart + elementCount < indEnd){
--- End diff --
try to reuse `indStart + elementCount`
~~~
var i = indStart
while (i < indEnd) {
...
...
i += 1
}
~~
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-55383567
test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-56136224
LGTM. I'm merging this into master. (We might need to make slight changes to some methods before the 1.2 release, but let's not block the multi-model training PR for now.)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17255323
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala ---
@@ -63,7 +88,162 @@ class DenseMatrix(val numRows: Int, val numCols: Int, val values: Array[Double])
override def toArray: Array[Double] = values
- private[mllib] override def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+ private [mllib] def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+
+ private [mllib] def apply(i: Int): Double = values(i)
+
+ private [mllib] def apply(r: Int, c: Int): Double = values(index(r, c))
+
+ private [mllib] def index(r: Int, c: Int): Int = r + numRows * c
+
+ private [mllib] def update(r: Int, c: Int, v: Double){
+ values(index(r, c)) = v
+ }
+
+ def copy = new DenseMatrix(numRows, numCols, values.clone())
+}
+
+/**
+ * Factory methods for [[org.apache.spark.mllib.linalg.DenseMatrix]].
+ *
+ * These methods can be used to generate common matrix types such as the Identity matrix, any
+ * diagonal matrix, zero matrix, and random matrices.
+ */
+object DenseMatrix {
+
+ /**
+ * Generate a `DenseMatrix` consisting of zeros.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values of zeros
+ */
+ def zeros(numRows: Int, numCols: Int) =
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(0.0))
+
+ /**
+ * Generate a `DenseMatrix` consisting of ones.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values of ones
+ */
+ def ones(numRows: Int, numCols: Int) =
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(1.0))
+
+ /**
+ * Generate an Identity Matrix in `DenseMatrix` format.
+ * @param n number of rows and columns of the matrix
+ * @return `DenseMatrix` with size `n` x `n` and values of ones on the diagonal
+ */
+ def eye(n: Int) = {
+ val identity = DenseMatrix.zeros(n,n)
+ for (i <- 0 until n){
+ identity.update(i, i, 1.0)
+ }
+ identity
+ }
+
+ /**
+ * Generate a `DenseMatrix` consisting of i.i.d. uniform random numbers.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values in U(0, 1)
+ */
+ def rand(numRows: Int, numCols: Int) = {
+ val rand = new scala.util.Random
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(rand.nextDouble()))
+ }
+
+ /**
+ * Generate a `DenseMatrix` consisting of i.i.d. gaussian random numbers.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values in N(0, 1)
+ */
+ def randn(numRows: Int, numCols: Int) = {
+ val rand = new scala.util.Random
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(rand.nextGaussian()))
+ }
+
+ /**
+ * Generate a diagonal matrix in `DenseMatrix` format from the supplied values.
+ * @param values values on the diagonal of the matrix
+ * @return Square `DenseMatrix` with size `values.length` x `values.length` and `values`
+ * on the diagonal
+ */
+ def diag(values: Array[Double]) = {
--- End diff --
Accept a `Vector` instead of `Array[Double]`?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17709096
--- Diff: mllib/src/test/scala/org/apache/spark/mllib/linalg/BreezeMatrixConversionSuite.scala ---
@@ -37,4 +37,26 @@ class BreezeMatrixConversionSuite extends FunSuite {
assert(mat.numCols === breeze.cols)
assert(mat.values.eq(breeze.data), "should not copy data")
}
+
+ test("sparse matrix to breeze") {
+ val values = Array(1.0, 2.0, 4.0, 5.0)
+ val colPointers = Array(0, 2, 4)
+ val rowIndices = Array(1, 2, 1, 2)
+ val mat = Matrices.sparse(3, 2, colPointers, rowIndices, values)
+ val breeze = mat.toBreeze.asInstanceOf[BSM[Double]]
+ assert(breeze.rows === mat.numRows)
+ assert(breeze.cols === mat.numCols)
+ assert(breeze.data.eq(mat.asInstanceOf[SparseMatrix].values), "should not copy data")
+ }
+
+ test("sparse breeze matrix to sparse matrix") {
+ val values = Array(1.0, 2.0, 4.0, 5.0)
+ val colPointers = Array(0, 2, 4)
--- End diff --
`colPtrs`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17709106
--- Diff: mllib/src/test/scala/org/apache/spark/mllib/linalg/MatricesSuite.scala ---
@@ -36,4 +36,79 @@ class MatricesSuite extends FunSuite {
Matrices.dense(3, 2, Array(0.0, 1.0, 2.0))
}
}
+
+ test("sparse matrix construction") {
+ val m = 3
+ val n = 2
+ val values = Array(1.0, 2.0, 4.0, 5.0)
+ val colIndices = Array(0, 2, 4)
+ val rowIndices = Array(1, 2, 1, 2)
+ val mat = Matrices.sparse(m, n, colIndices, rowIndices, values).asInstanceOf[SparseMatrix]
+ assert(mat.numRows === m)
+ assert(mat.numCols === n)
+ assert(mat.values.eq(values), "should not copy data")
+ }
+
+ test("sparse matrix construction with wrong number of elements") {
+ intercept[RuntimeException] {
+ Matrices.sparse(3, 2, Array(0, 1), Array(1, 2, 1), Array(0.0, 1.0, 2.0))
+ }
+
+ intercept[RuntimeException] {
+ Matrices.sparse(3, 2, Array(0, 1, 2), Array(1, 2), Array(0.0, 1.0, 2.0))
+ }
+ }
+
+ test("matrix copies are deep copies") {
+ val m = 3
+ val n = 2
+
+ val denseMat = Matrices.dense(m, n, Array(0.0, 1.0, 2.0, 3.0, 4.0, 5.0))
+ val denseCopy = denseMat.copy
+
+ assert(!denseMat.toArray.eq(denseCopy.toArray))
+
+ val values = Array(1.0, 2.0, 4.0, 5.0)
+ val colIndices = Array(0, 2, 4)
+ val rowIndices = Array(1, 2, 1, 2)
+ val sparseMat = Matrices.sparse(m, n, colIndices, rowIndices, values)
+ val sparseCopy = sparseMat.copy
+
+ assert(!sparseMat.toArray.eq(sparseCopy.toArray))
+ }
+
+ test("matrix indexing and updating") {
+ val m = 3
+ val n = 2
+ val allValues = Array(0.0, 1.0, 2.0, 3.0, 4.0, 0.0)
+
+ val denseMat = new DenseMatrix(m, n, allValues)
+
+ assert(denseMat(0, 1) == 3.0)
+ assert(denseMat(0, 1) == denseMat.values(3))
+ assert(denseMat(0, 1) == denseMat(3))
+ assert(denseMat(0, 0) == 0.0)
+
+ denseMat.update(0, 0, 10.0)
+ assert(denseMat(0, 0) == 10.0)
+ assert(denseMat.values(0) == 10.0)
+
+ val sparseValues = Array(1.0, 2.0, 3.0, 4.0)
+ val colIndices = Array(0, 2, 4)
+ val rowIndices = Array(1, 2, 0, 1)
+ val sparseMat = new SparseMatrix(m, n, colIndices, rowIndices, sparseValues)
+
+ assert(sparseMat(0, 1) == 3.0)
+ assert(sparseMat(0, 1) == sparseMat.values(2))
+ assert(sparseMat(0, 1) == sparseMat(2))
+ assert(sparseMat(0, 0) == 0.0)
+
+ intercept[IllegalArgumentException] {
+ sparseMat.update(0, 0, 10.0)
+ }
+
+ sparseMat.update(0, 1, 10.0)
+ assert(sparseMat(0, 1) == 10.0)
--- End diff --
`==` -> `===` (please update others as well)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by brkyvz <gi...@git.apache.org>.
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-54696200
The following tests were run on a Mac OS X 10.9.3
2.8 GHz Intel Core i7
8 GB 1600 MHz DDR3.
Hyper-threading was disabled and turbo boost was turned off. Effective numRows or Effective numCols stand for the effective number of rows or columns that are used in the computation. As the transpose multiplications are also used, that means that the number of rows of A, which has dimensions `m x n` may be `m`, but the effective number of rows are `n`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17709088
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala ---
@@ -59,11 +93,113 @@ trait Matrix extends Serializable {
*/
class DenseMatrix(val numRows: Int, val numCols: Int, val values: Array[Double]) extends Matrix {
- require(values.length == numRows * numCols)
+ require(values.length == numRows * numCols, "The number of values supplied doesn't match the " +
+ s"size of the matrix! values.length: ${values.length}, numRows * numCols: ${numRows * numCols}")
override def toArray: Array[Double] = values
- private[mllib] override def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+ private[mllib] def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+
+ private[mllib] def apply(i: Int): Double = values(i)
+
+ private[mllib] def apply(i: Int, j: Int): Double = values(index(i, j))
+
+ private[mllib] def index(i: Int, j: Int): Int = i + numRows * j
+
+ private[mllib] def update(i: Int, j: Int, v: Double){
+ values(index(i, j)) = v
+ }
+
+ override def copy = new DenseMatrix(numRows, numCols, values.clone())
+}
+
+/**
+ * Column-majored sparse matrix.
+ * The entry values are stored in Compressed Sparse Column (CSC) format.
+ * For example, the following matrix
+ * {{{
+ * 1.0 0.0 4.0
+ * 0.0 3.0 5.0
+ * 2.0 0.0 6.0
+ * }}}
+ * is stored as `values: [1.0, 2.0, 3.0, 4.0, 5.0, 6.0]`,
+ * `rowIndices=[0, 2, 1, 0, 1, 2]`, `colPointers=[0, 2, 3, 6]`.
+ *
+ * @param numRows number of rows
+ * @param numCols number of columns
+ * @param colPtrs the index corresponding to the start of a new column
+ * @param rowIndices the row index of the entry
+ * @param values non-zero matrix entries in column major
+ */
+class SparseMatrix(
+ val numRows: Int,
+ val numCols: Int,
+ val colPtrs: Array[Int],
+ val rowIndices: Array[Int],
+ val values: Array[Double]) extends Matrix {
+
+ require(values.length == rowIndices.length, "The number of row indices and values don't match! " +
+ s"values.length: ${values.length}, rowIndices.length: ${rowIndices.length}")
+ require(colPtrs.length == numCols + 1, "The length of the column indices should be the " +
+ s"number of columns + 1. Currently, colPointers.length: ${colPtrs.length}, " +
+ s"numCols: $numCols")
+
+ override def toArray: Array[Double] = {
+ val arr = Array.fill(numRows * numCols)(0.0)
+ var j = 0
+ while (j < numCols) {
+ var i = colPtrs(j)
+ val indEnd = colPtrs(j + 1)
+ val offset = j * numRows
+ while (i < indEnd) {
+ val rowIndex = rowIndices(i)
+ arr(offset + rowIndex) = values(i)
+ i += 1
+ }
+ j += 1
+ }
+ arr
+ }
+
+ private[mllib] def toBreeze: BM[Double] =
+ new BSM[Double](values, numRows, numCols, colPtrs, rowIndices)
+
+ private[mllib] def apply(i: Int): Double = values(i)
+
+ private[mllib] def apply(i: Int, j: Int): Double = {
+ val ind = index(i, j)
+ if (ind == -1) 0.0 else values(ind)
+ }
+
+ private[mllib] def index(i: Int, j: Int): Int = {
+ var regionStart = colPtrs(j)
+ var regionEnd = colPtrs(j + 1)
+ while (regionStart <= regionEnd) {
+ val mid = (regionStart + regionEnd) / 2
+ if (rowIndices(mid) == i){
+ return mid
+ } else if (regionStart == regionEnd) {
+ return -1
+ } else if (rowIndices(mid) > i) {
+ regionEnd = mid
+ } else {
+ regionStart = mid
+ }
+ }
+ -1
+ }
+
+ private[mllib] def update(i: Int, j: Int, v: Double){
--- End diff --
`: Unit = {`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17709059
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +201,368 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ if (alpha == 0.0) {
+ logWarning("gemm: alpha is equal to 0. Returning C.")
+ } else {
+ A match {
+ case sparse: SparseMatrix =>
+ gemm(transA, transB, alpha, sparse, B, beta, C)
+ case dense: DenseMatrix =>
+ gemm(transA, transB, alpha, dense, B, beta, C)
+ case _ =>
+ throw new IllegalArgumentException(s"gemm doesn't support matrix type ${A.getClass}.")
+ }
+ }
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ gemm(false, false, alpha, A, B, beta, C)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `DenseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: DenseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+ val tAstr = if (!transA) "N" else "T"
+ val tBstr = if (!transB) "N" else "T"
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ nativeBLAS.dgemm(tAstr, tBstr, mA, nB, kA, alpha, A.values, A.numRows, B.values, B.numRows,
+ beta, C.values, C.numRows)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `SparseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: SparseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ val Avals = A.values
+ val Arows = if (!transA) A.rowIndices else A.colPtrs
+ val Acols = if (!transA) A.colPtrs else A.rowIndices
+
+ // Slicing is easy in this case. This is the optimal multiplication setting for sparse matrices
+ if (transA){
+ var colCounterForB = 0
+ if (!transB){ // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var rowCounterForA = 0
+ val Cstart = colCounterForB * mA
+ val Bstart = colCounterForB * kA
+ while (rowCounterForA < mA) {
+ var i = Arows(rowCounterForA)
+ val indEnd = Arows(rowCounterForA + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B.values(Bstart + Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounterForA
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ } else {
+ while (colCounterForB < nB) {
+ var rowCounter = 0
+ val Cstart = colCounterForB * mA
+ while (rowCounter < mA) {
+ var i = Arows(rowCounter)
+ val indEnd = Arows(rowCounter + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B(colCounterForB, Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounter
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounter += 1
+ }
+ colCounterForB += 1
+ }
+ }
+ } else {
+ // Scale matrix first if `beta` is not equal to 0.0
+ if (beta != 0.0){
+ nativeBLAS.dscal(C.values.length, beta, C.values, 1)
+ }
+ // Perform matrix multiplication and add to C. The rows of A are multiplied by the columns of
+ // B, and added to C.
+ var colCounterForB = 0 // the column to be updated in C
+ if (!transB) { // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var colCounterForA = 0 // The column of A to multiply with the row of B
+ while (colCounterForA < kA){
+ var i = Acols(colCounterForA)
+ val indEnd = Acols(colCounterForA + 1)
+ val Bval = B(colCounterForA, colCounterForB)
+ val Cstart = colCounterForB * mA
+ while (i < indEnd){
+ C.values(Cstart + Arows(i)) += Avals(i) * Bval * alpha
+ i += 1
+ }
+ colCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ } else {
+ while (colCounterForB < nB) {
+ var colCounterForA = 0 // The column of A to multiply with the row of B
+ while (colCounterForA < kA){
+ var i = Acols(colCounterForA)
+ val indEnd = Acols(colCounterForA + 1)
+ val Bval = B(colCounterForB, colCounterForA)
+ val Cstart = colCounterForB * mA
+ while (i < indEnd){
+ C.values(Cstart + Arows(i)) += Avals(i) * Bval * alpha
+ i += 1
+ }
+ colCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ }
+ }
+ }
+
+ /**
+ * y := alpha * A * x + beta * y
+ * @param trans specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ * @param beta a scalar that can be used to scale vector y.
+ * @param y the resulting vector y. Size of m x 1.
+ */
+ def gemv(
+ trans: Boolean,
+ alpha: Double,
+ A: Matrix,
+ x: DenseVector,
+ beta: Double,
+ y: DenseVector): Unit = {
+
+ val mA: Int = if (!trans) A.numRows else A.numCols
+ val nx: Int = x.size
+ val nA: Int = if (!trans) A.numCols else A.numRows
+
+ require(nA == nx, s"The columns of A don't match the number of elements of x. A: $nA, x: $nx")
+ require(mA == y.size,
+ s"The rows of A don't match the number of elements of y. A: $mA, y:${y.size}}")
+ if (alpha == 0.0) {
+ logWarning("gemv: alpha is equal to 0. Returning y.")
+ } else {
+ A match {
+ case sparse: SparseMatrix =>
+ gemv(trans, alpha, sparse, x, beta, y)
+ case dense: DenseMatrix =>
+ gemv(trans, alpha, dense, x, beta, y)
+ case _ =>
+ throw new IllegalArgumentException(s"gemv doesn't support matrix type ${A.getClass}.")
+ }
+ }
+ }
+
+ /**
+ * y := alpha * A * x + beta * y
+ *
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ * @param beta a scalar that can be used to scale vector y.
+ * @param y the resulting vector y. Size of m x 1.
+ */
+ def gemv(
+ alpha: Double,
+ A: Matrix,
+ x: DenseVector,
+ beta: Double,
+ y: DenseVector): Unit = {
+ gemv(false, alpha, A, x, beta, y)
+ }
+
+ /**
+ * y := alpha * A * x
+ *
+ * @param trans specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ *
+ * @return `DenseVector` y, the result of the matrix-vector multiplication. Size of m x 1.
+ */
+ def gemv(
--- End diff --
All BLAS routines return `Unit`. We can implement this inside `A.times(b)`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17709093
--- Diff: mllib/src/test/scala/org/apache/spark/mllib/linalg/BLASSuite.scala ---
@@ -126,4 +126,116 @@ class BLASSuite extends FunSuite {
}
}
}
+
+ test("gemm") {
+
+ val dA =
+ new DenseMatrix(4, 3, Array(0.0, 1.0, 0.0, 0.0, 2.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 3.0))
+ val sA = new SparseMatrix(4, 3, Array(0, 1, 3, 4), Array(1, 0, 2, 3), Array(1.0, 2.0, 1.0, 3.0))
+
+ val B = new DenseMatrix(3, 2, Array(1.0, 0.0, 0.0, 0.0, 2.0, 1.0))
+ val expected = new DenseMatrix(4, 2, Array(0.0, 1.0, 0.0, 0.0, 4.0, 0.0, 2.0, 3.0))
+
+ assert(dA times B ~== expected absTol 1e-15)
+ assert(sA times B ~== expected absTol 1e-15)
+
+ val C1 = new DenseMatrix(4, 2, Array(1.0, 0.0, 2.0, 1.0, 0.0, 0.0, 1.0, 0.0))
+ val C2 = C1.copy
+ val C3 = C1.copy
+ val C4 = C1.copy
+ val C5 = C1.copy
+ val C6 = C1.copy
+ val C7 = C1.copy
+ val C8 = C1.copy
+ val expected2 = new DenseMatrix(4, 2, Array(2.0, 1.0, 4.0, 2.0, 4.0, 0.0, 4.0, 3.0))
+ val expected3 = new DenseMatrix(4, 2, Array(2.0, 2.0, 4.0, 2.0, 8.0, 0.0, 6.0, 6.0))
+
+ gemm(1.0, dA, B, 2.0, C1)
+ gemm(1.0, sA, B, 2.0, C2)
+ gemm(2.0, dA, B, 2.0, C3)
+ gemm(2.0, sA, B, 2.0, C4)
+ assert(C1 ~== expected2 absTol 1e-15)
+ assert(C2 ~== expected2 absTol 1e-15)
+ assert(C3 ~== expected3 absTol 1e-15)
+ assert(C4 ~== expected3 absTol 1e-15)
+
+ withClue("columns of A don't match the rows of B") {
+ intercept[Exception] {
+ gemm(true, false, 1.0, dA, B, 2.0, C1)
+ }
+ }
+
+ val dAT =
+ new DenseMatrix(3, 4, Array(0.0, 2.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 3.0))
+ val sAT =
+ new SparseMatrix(3, 4, Array(0, 1, 2, 3, 4), Array(1, 0, 1, 2), Array(2.0, 1.0, 1.0, 3.0))
+
+ assert(dAT transposeTimes B ~== expected absTol 1e-15)
+ assert(sAT transposeTimes B ~== expected absTol 1e-15)
+
+ gemm(true, false, 1.0, dAT, B, 2.0, C5)
+ gemm(true, false, 1.0, sAT, B, 2.0, C6)
+ gemm(true, false, 2.0, dAT, B, 2.0, C7)
+ gemm(true, false, 2.0, sAT, B, 2.0, C8)
+ assert(C5 ~== expected2 absTol 1e-15)
+ assert(C6 ~== expected2 absTol 1e-15)
+ assert(C7 ~== expected3 absTol 1e-15)
+ assert(C8 ~== expected3 absTol 1e-15)
+
--- End diff --
remove extra empty line
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17709067
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +201,368 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ if (alpha == 0.0) {
+ logWarning("gemm: alpha is equal to 0. Returning C.")
+ } else {
+ A match {
+ case sparse: SparseMatrix =>
+ gemm(transA, transB, alpha, sparse, B, beta, C)
+ case dense: DenseMatrix =>
+ gemm(transA, transB, alpha, dense, B, beta, C)
+ case _ =>
+ throw new IllegalArgumentException(s"gemm doesn't support matrix type ${A.getClass}.")
+ }
+ }
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ gemm(false, false, alpha, A, B, beta, C)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `DenseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: DenseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+ val tAstr = if (!transA) "N" else "T"
+ val tBstr = if (!transB) "N" else "T"
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ nativeBLAS.dgemm(tAstr, tBstr, mA, nB, kA, alpha, A.values, A.numRows, B.values, B.numRows,
+ beta, C.values, C.numRows)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `SparseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: SparseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ val Avals = A.values
+ val Arows = if (!transA) A.rowIndices else A.colPtrs
+ val Acols = if (!transA) A.colPtrs else A.rowIndices
+
+ // Slicing is easy in this case. This is the optimal multiplication setting for sparse matrices
+ if (transA){
+ var colCounterForB = 0
+ if (!transB){ // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var rowCounterForA = 0
+ val Cstart = colCounterForB * mA
+ val Bstart = colCounterForB * kA
+ while (rowCounterForA < mA) {
+ var i = Arows(rowCounterForA)
+ val indEnd = Arows(rowCounterForA + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B.values(Bstart + Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounterForA
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ } else {
+ while (colCounterForB < nB) {
+ var rowCounter = 0
+ val Cstart = colCounterForB * mA
+ while (rowCounter < mA) {
+ var i = Arows(rowCounter)
+ val indEnd = Arows(rowCounter + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B(colCounterForB, Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounter
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounter += 1
+ }
+ colCounterForB += 1
+ }
+ }
+ } else {
+ // Scale matrix first if `beta` is not equal to 0.0
+ if (beta != 0.0){
+ nativeBLAS.dscal(C.values.length, beta, C.values, 1)
+ }
+ // Perform matrix multiplication and add to C. The rows of A are multiplied by the columns of
+ // B, and added to C.
+ var colCounterForB = 0 // the column to be updated in C
+ if (!transB) { // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var colCounterForA = 0 // The column of A to multiply with the row of B
+ while (colCounterForA < kA){
+ var i = Acols(colCounterForA)
+ val indEnd = Acols(colCounterForA + 1)
+ val Bval = B(colCounterForA, colCounterForB)
+ val Cstart = colCounterForB * mA
+ while (i < indEnd){
+ C.values(Cstart + Arows(i)) += Avals(i) * Bval * alpha
+ i += 1
+ }
+ colCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ } else {
+ while (colCounterForB < nB) {
+ var colCounterForA = 0 // The column of A to multiply with the row of B
+ while (colCounterForA < kA){
+ var i = Acols(colCounterForA)
+ val indEnd = Acols(colCounterForA + 1)
+ val Bval = B(colCounterForB, colCounterForA)
+ val Cstart = colCounterForB * mA
+ while (i < indEnd){
+ C.values(Cstart + Arows(i)) += Avals(i) * Bval * alpha
+ i += 1
+ }
+ colCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ }
+ }
+ }
+
+ /**
+ * y := alpha * A * x + beta * y
+ * @param trans specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ * @param beta a scalar that can be used to scale vector y.
+ * @param y the resulting vector y. Size of m x 1.
+ */
+ def gemv(
+ trans: Boolean,
+ alpha: Double,
+ A: Matrix,
+ x: DenseVector,
+ beta: Double,
+ y: DenseVector): Unit = {
+
+ val mA: Int = if (!trans) A.numRows else A.numCols
+ val nx: Int = x.size
+ val nA: Int = if (!trans) A.numCols else A.numRows
+
+ require(nA == nx, s"The columns of A don't match the number of elements of x. A: $nA, x: $nx")
+ require(mA == y.size,
+ s"The rows of A don't match the number of elements of y. A: $mA, y:${y.size}}")
+ if (alpha == 0.0) {
+ logWarning("gemv: alpha is equal to 0. Returning y.")
+ } else {
+ A match {
+ case sparse: SparseMatrix =>
+ gemv(trans, alpha, sparse, x, beta, y)
+ case dense: DenseMatrix =>
+ gemv(trans, alpha, dense, x, beta, y)
+ case _ =>
+ throw new IllegalArgumentException(s"gemv doesn't support matrix type ${A.getClass}.")
+ }
+ }
+ }
+
+ /**
+ * y := alpha * A * x + beta * y
+ *
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ * @param beta a scalar that can be used to scale vector y.
+ * @param y the resulting vector y. Size of m x 1.
+ */
+ def gemv(
+ alpha: Double,
+ A: Matrix,
+ x: DenseVector,
+ beta: Double,
+ y: DenseVector): Unit = {
+ gemv(false, alpha, A, x, beta, y)
+ }
+
+ /**
+ * y := alpha * A * x
+ *
+ * @param trans specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ *
+ * @return `DenseVector` y, the result of the matrix-vector multiplication. Size of m x 1.
+ */
+ def gemv(
+ trans: Boolean,
+ alpha: Double,
+ A: Matrix,
+ x: DenseVector): DenseVector = {
+ val m = if(!trans) A.numRows else A.numCols
+
+ val y: DenseVector = new DenseVector(Array.fill(m)(0.0))
+ gemv(trans, alpha, A, x, 0.0, y)
+
+ y
+ }
+
+ /**
+ * y := alpha * A * x
+ *
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ *
+ * @return `DenseVector` y, the result of the matrix-vector multiplication. Size of m x 1.
+ */
+ def gemv(
+ alpha: Double,
+ A: Matrix,
+ x: DenseVector): DenseVector = {
+ gemv(false, alpha, A, x)
+ }
+
+
+ /**
+ * y := alpha * A * x + beta * y
+ * For `DenseMatrix` A.
+ */
+ private def gemv(
+ trans: Boolean,
+ alpha: Double,
+ A: DenseMatrix,
+ x: DenseVector,
+ beta: Double,
+ y: DenseVector): Unit = {
+ val tStrA = if (!trans) "N" else "T"
+ nativeBLAS.dgemv(tStrA, A.numRows, A.numCols, alpha, A.values, A.numRows, x.values, 1, beta,
+ y.values, 1)
+ }
+
+ /**
+ * y := alpha * A * x + beta * y
+ * For `SparseMatrix` A.
+ */
+ private def gemv(
+ trans: Boolean,
+ alpha: Double,
+ A: SparseMatrix,
+ x: DenseVector,
+ beta: Double,
+ y: DenseVector): Unit = {
+
+ val mA: Int = if(!trans) A.numRows else A.numCols
+ val nA: Int = if(!trans) A.numCols else A.numRows
+
+ val Avals = A.values
+ val Arows = if (!trans) A.rowIndices else A.colPtrs
+ val Acols = if (!trans) A.colPtrs else A.rowIndices
+
+ // Slicing is easy in this case. This is the optimal multiplication setting for sparse matrices
+ if (trans){
+ var rowCounter = 0
+ while (rowCounter < mA){
+ var i = Arows(rowCounter)
+ var sum = 0.0
+ while(i < Arows(rowCounter + 1)){
--- End diff --
cache `Arows(rowCounter + 1)`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17709081
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala ---
@@ -59,11 +93,113 @@ trait Matrix extends Serializable {
*/
class DenseMatrix(val numRows: Int, val numCols: Int, val values: Array[Double]) extends Matrix {
- require(values.length == numRows * numCols)
+ require(values.length == numRows * numCols, "The number of values supplied doesn't match the " +
+ s"size of the matrix! values.length: ${values.length}, numRows * numCols: ${numRows * numCols}")
override def toArray: Array[Double] = values
- private[mllib] override def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+ private[mllib] def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+
+ private[mllib] def apply(i: Int): Double = values(i)
+
+ private[mllib] def apply(i: Int, j: Int): Double = values(index(i, j))
+
+ private[mllib] def index(i: Int, j: Int): Int = i + numRows * j
+
+ private[mllib] def update(i: Int, j: Int, v: Double){
+ values(index(i, j)) = v
+ }
+
+ override def copy = new DenseMatrix(numRows, numCols, values.clone())
+}
+
+/**
+ * Column-majored sparse matrix.
+ * The entry values are stored in Compressed Sparse Column (CSC) format.
+ * For example, the following matrix
+ * {{{
+ * 1.0 0.0 4.0
+ * 0.0 3.0 5.0
+ * 2.0 0.0 6.0
+ * }}}
+ * is stored as `values: [1.0, 2.0, 3.0, 4.0, 5.0, 6.0]`,
+ * `rowIndices=[0, 2, 1, 0, 1, 2]`, `colPointers=[0, 2, 3, 6]`.
+ *
+ * @param numRows number of rows
+ * @param numCols number of columns
+ * @param colPtrs the index corresponding to the start of a new column
+ * @param rowIndices the row index of the entry
--- End diff --
Let's write down the contract. `rowIndices` should be strictly increasing for each column.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17255344
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala ---
@@ -63,7 +88,162 @@ class DenseMatrix(val numRows: Int, val numCols: Int, val values: Array[Double])
override def toArray: Array[Double] = values
- private[mllib] override def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+ private [mllib] def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+
+ private [mllib] def apply(i: Int): Double = values(i)
+
+ private [mllib] def apply(r: Int, c: Int): Double = values(index(r, c))
+
+ private [mllib] def index(r: Int, c: Int): Int = r + numRows * c
+
+ private [mllib] def update(r: Int, c: Int, v: Double){
+ values(index(r, c)) = v
+ }
+
+ def copy = new DenseMatrix(numRows, numCols, values.clone())
+}
+
+/**
+ * Factory methods for [[org.apache.spark.mllib.linalg.DenseMatrix]].
+ *
+ * These methods can be used to generate common matrix types such as the Identity matrix, any
+ * diagonal matrix, zero matrix, and random matrices.
+ */
+object DenseMatrix {
+
+ /**
+ * Generate a `DenseMatrix` consisting of zeros.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values of zeros
+ */
+ def zeros(numRows: Int, numCols: Int) =
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(0.0))
+
+ /**
+ * Generate a `DenseMatrix` consisting of ones.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values of ones
+ */
+ def ones(numRows: Int, numCols: Int) =
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(1.0))
+
+ /**
+ * Generate an Identity Matrix in `DenseMatrix` format.
+ * @param n number of rows and columns of the matrix
+ * @return `DenseMatrix` with size `n` x `n` and values of ones on the diagonal
+ */
+ def eye(n: Int) = {
+ val identity = DenseMatrix.zeros(n,n)
+ for (i <- 0 until n){
+ identity.update(i, i, 1.0)
+ }
+ identity
+ }
+
+ /**
+ * Generate a `DenseMatrix` consisting of i.i.d. uniform random numbers.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values in U(0, 1)
+ */
+ def rand(numRows: Int, numCols: Int) = {
+ val rand = new scala.util.Random
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(rand.nextDouble()))
+ }
+
+ /**
+ * Generate a `DenseMatrix` consisting of i.i.d. gaussian random numbers.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values in N(0, 1)
+ */
+ def randn(numRows: Int, numCols: Int) = {
+ val rand = new scala.util.Random
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(rand.nextGaussian()))
+ }
+
+ /**
+ * Generate a diagonal matrix in `DenseMatrix` format from the supplied values.
+ * @param values values on the diagonal of the matrix
+ * @return Square `DenseMatrix` with size `values.length` x `values.length` and `values`
+ * on the diagonal
+ */
+ def diag(values: Array[Double]) = {
+ val n = values.length
+ val matrix = DenseMatrix.eye(n)
+ for (i <- 0 until n) matrix.update(i, i, values(i))
+ matrix
+ }
+}
+
+/**
+ * Column-majored sparse matrix.
+ * The entry values are stored in Compressed-Column Storage (CCS) format.
+ * For example, the following matrix
+ * {{{
+ * 1.0 0.0 4.0
+ * 0.0 3.0 5.0
+ * 2.0 0.0 6.0
+ * }}}
+ * is stored as `values: [1.0, 2.0, 3.0, 4.0, 5.0, 6.0]`,
+ * `rowIndices=[0, 2, 1, 0, 1, 2]`, `colPointers=[0, 2, 3, 6]`.
+ *
+ * @param numRows number of rows
+ * @param numCols number of columns
+ * @param colPointers the index corresponding to the start of a new column
+ * @param rowIndices the row index of the entry
+ * @param values non-zero matrix entries in column major
+ */
+class SparseMatrix(
+ val numRows: Int,
+ val numCols: Int,
+ val colPointers: Array[Int],
+ val rowIndices: Array[Int],
+ val values: Array[Double]) extends Matrix {
+
+ require(values.length == rowIndices.length, "The number of row indices and values don't match!")
+ require(colPointers.length == numCols + 1, "The length of the column indices should be the " +
+ s"number of columns + 1. Currently, colPointers.length: ${colPointers.length}, " +
+ s"numCols: $numCols")
+
+ override def toArray: Array[Double] = values
+
+ private [mllib] def toBreeze: BM[Double] =
+ new BSM[Double](values, numRows, numCols, colPointers, rowIndices)
+
+ private [mllib] def apply(i: Int): Double = values(i)
+
+ private [mllib] def apply(r: Int, c: Int): Double = {
+ val ind = index(r,c)
+ if (ind == -1) 0.0 else values(ind)
+ }
+
+ private [mllib] def index(r: Int, c: Int): Int = {
+ val regionStart = colPointers(c)
+ val regionEnd = colPointers(c + 1)
+ val region = rowIndices.slice(regionStart, regionEnd)
+ if (region.contains(r)){
--- End diff --
use binary search?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by ScrapCodes <gi...@git.apache.org>.
Github user ScrapCodes commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-56118218
Can you rebase to the tip of master and add tests in version 1.2 section:
https://github.com/apache/spark/blob/master/project/MimaExcludes.scala#L37
instead of 1.1.
Prashant Sharma
On Fri, Sep 19, 2014 at 4:42 AM, Burak Yavuz <no...@github.com>
wrote:
> @ScrapCodes <https://github.com/ScrapCodes>, I have some mima
> incompatibility issues here. I've added a few methods to the trait Matrix
> in MLlib, and that causes a MissingMethodProblem. I sealed the trait, and
> added the exclude statements to the MimaExcludes, but the tests still fail.
> dev/mima passes on my local machine.
> Do you have any idea of what might be going on? The commits d162684
> <https://github.com/apache/spark/commit/d16268496cefe06d8818b20721eede56a9de41a2>
> and
> 272feb9
> <https://github.com/apache/spark/commit/272feb9f63517c52e1991988b4fbd8869a992dc4>
> were to address this issue.
>
> Can you please take a look?
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/apache/spark/pull/2294#issuecomment-56116462>.
>
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17709076
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala ---
@@ -36,9 +37,42 @@ trait Matrix extends Serializable {
/** Converts to a breeze matrix. */
private[mllib] def toBreeze: BM[Double]
+ /** Gets the i-th element in the array backing the matrix. */
+ private[mllib] def apply(i: Int): Double
+
/** Gets the (i, j)-th element. */
- private[mllib] def apply(i: Int, j: Int): Double = toBreeze(i, j)
+ private[mllib] def apply(i: Int, j: Int): Double
+
+ /** Return the index for the (i, j)-th element in the backing array. */
+ private[mllib] def index(i: Int, j: Int): Int
+
+ /** Update element at (i, j) */
+ private[mllib] def update(i: Int, j: Int, v: Double)
+
+ /** Get a deep copy of the matrix. */
+ def copy: Matrix
+
+ /** Convenience method for `Matrix`-`DenseMatrix` multiplication. */
+ def times(y: DenseMatrix): DenseMatrix = {
--- End diff --
We use `multiply` in distributed matrix and we should be consistent.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17709104
--- Diff: mllib/src/test/scala/org/apache/spark/mllib/linalg/MatricesSuite.scala ---
@@ -36,4 +36,79 @@ class MatricesSuite extends FunSuite {
Matrices.dense(3, 2, Array(0.0, 1.0, 2.0))
}
}
+
+ test("sparse matrix construction") {
+ val m = 3
+ val n = 2
+ val values = Array(1.0, 2.0, 4.0, 5.0)
+ val colIndices = Array(0, 2, 4)
+ val rowIndices = Array(1, 2, 1, 2)
+ val mat = Matrices.sparse(m, n, colIndices, rowIndices, values).asInstanceOf[SparseMatrix]
+ assert(mat.numRows === m)
+ assert(mat.numCols === n)
+ assert(mat.values.eq(values), "should not copy data")
+ }
+
+ test("sparse matrix construction with wrong number of elements") {
+ intercept[RuntimeException] {
+ Matrices.sparse(3, 2, Array(0, 1), Array(1, 2, 1), Array(0.0, 1.0, 2.0))
+ }
+
+ intercept[RuntimeException] {
+ Matrices.sparse(3, 2, Array(0, 1, 2), Array(1, 2), Array(0.0, 1.0, 2.0))
+ }
+ }
+
+ test("matrix copies are deep copies") {
+ val m = 3
+ val n = 2
+
+ val denseMat = Matrices.dense(m, n, Array(0.0, 1.0, 2.0, 3.0, 4.0, 5.0))
+ val denseCopy = denseMat.copy
+
+ assert(!denseMat.toArray.eq(denseCopy.toArray))
+
+ val values = Array(1.0, 2.0, 4.0, 5.0)
+ val colIndices = Array(0, 2, 4)
+ val rowIndices = Array(1, 2, 1, 2)
+ val sparseMat = Matrices.sparse(m, n, colIndices, rowIndices, values)
+ val sparseCopy = sparseMat.copy
+
+ assert(!sparseMat.toArray.eq(sparseCopy.toArray))
+ }
+
+ test("matrix indexing and updating") {
+ val m = 3
+ val n = 2
+ val allValues = Array(0.0, 1.0, 2.0, 3.0, 4.0, 0.0)
+
+ val denseMat = new DenseMatrix(m, n, allValues)
+
+ assert(denseMat(0, 1) == 3.0)
+ assert(denseMat(0, 1) == denseMat.values(3))
+ assert(denseMat(0, 1) == denseMat(3))
+ assert(denseMat(0, 0) == 0.0)
+
+ denseMat.update(0, 0, 10.0)
+ assert(denseMat(0, 0) == 10.0)
+ assert(denseMat.values(0) == 10.0)
+
+ val sparseValues = Array(1.0, 2.0, 3.0, 4.0)
+ val colIndices = Array(0, 2, 4)
+ val rowIndices = Array(1, 2, 0, 1)
+ val sparseMat = new SparseMatrix(m, n, colIndices, rowIndices, sparseValues)
+
+ assert(sparseMat(0, 1) == 3.0)
+ assert(sparseMat(0, 1) == sparseMat.values(2))
+ assert(sparseMat(0, 1) == sparseMat(2))
+ assert(sparseMat(0, 0) == 0.0)
+
+ intercept[IllegalArgumentException] {
--- End diff --
Should throw `NoSuchElementException`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17704454
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +201,368 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ if (alpha == 0.0) {
+ logWarning("gemm: alpha is equal to 0. Returning C.")
+ } else {
+ A match {
+ case sparse: SparseMatrix =>
+ gemm(transA, transB, alpha, sparse, B, beta, C)
+ case dense: DenseMatrix =>
+ gemm(transA, transB, alpha, dense, B, beta, C)
+ case _ =>
+ throw new IllegalArgumentException(s"gemm doesn't support matrix type ${A.getClass}.")
+ }
+ }
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ gemm(false, false, alpha, A, B, beta, C)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `DenseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: DenseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+ val tAstr = if (!transA) "N" else "T"
+ val tBstr = if (!transB) "N" else "T"
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ nativeBLAS.dgemm(tAstr, tBstr, mA, nB, kA, alpha, A.values, A.numRows, B.values, B.numRows,
+ beta, C.values, C.numRows)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `SparseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: SparseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ val Avals = A.values
+ val Arows = if (!transA) A.rowIndices else A.colPtrs
+ val Acols = if (!transA) A.colPtrs else A.rowIndices
+
+ // Slicing is easy in this case. This is the optimal multiplication setting for sparse matrices
+ if (transA){
+ var colCounterForB = 0
+ if (!transB){ // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var rowCounterForA = 0
+ val Cstart = colCounterForB * mA
+ val Bstart = colCounterForB * kA
+ while (rowCounterForA < mA) {
+ var i = Arows(rowCounterForA)
+ val indEnd = Arows(rowCounterForA + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B.values(Bstart + Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounterForA
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ } else {
+ while (colCounterForB < nB) {
+ var rowCounter = 0
+ val Cstart = colCounterForB * mA
+ while (rowCounter < mA) {
+ var i = Arows(rowCounter)
+ val indEnd = Arows(rowCounter + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B(colCounterForB, Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounter
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounter += 1
+ }
+ colCounterForB += 1
+ }
+ }
+ } else {
+ // Scale matrix first if `beta` is not equal to 0.0
+ if (beta != 0.0){
+ nativeBLAS.dscal(C.values.length, beta, C.values, 1)
+ }
+ // Perform matrix multiplication and add to C. The rows of A are multiplied by the columns of
+ // B, and added to C.
+ var colCounterForB = 0 // the column to be updated in C
+ if (!transB) { // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var colCounterForA = 0 // The column of A to multiply with the row of B
+ while (colCounterForA < kA){
+ var i = Acols(colCounterForA)
+ val indEnd = Acols(colCounterForA + 1)
+ val Bval = B(colCounterForA, colCounterForB)
+ val Cstart = colCounterForB * mA
+ while (i < indEnd){
+ C.values(Cstart + Arows(i)) += Avals(i) * Bval * alpha
+ i += 1
+ }
+ colCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ } else {
+ while (colCounterForB < nB) {
+ var colCounterForA = 0 // The column of A to multiply with the row of B
+ while (colCounterForA < kA){
+ var i = Acols(colCounterForA)
+ val indEnd = Acols(colCounterForA + 1)
+ val Bval = B(colCounterForB, colCounterForA)
+ val Cstart = colCounterForB * mA
--- End diff --
ditto: move this to the outer loop
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-55986305
[QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20506/consoleFull) for PR 2294 at commit [`418def8`](https://github.com/apache/spark/commit/418def8e940b93f0d24e9b3158ecc0e130d16a83).
* This patch **fails** unit tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `sealed trait Matrix extends Serializable `
* `class SparseMatrix(`
* `sealed trait Vector extends Serializable `
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by brkyvz <gi...@git.apache.org>.
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17746583
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +201,368 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ if (alpha == 0.0) {
+ logWarning("gemm: alpha is equal to 0. Returning C.")
+ } else {
+ A match {
+ case sparse: SparseMatrix =>
+ gemm(transA, transB, alpha, sparse, B, beta, C)
+ case dense: DenseMatrix =>
+ gemm(transA, transB, alpha, dense, B, beta, C)
+ case _ =>
+ throw new IllegalArgumentException(s"gemm doesn't support matrix type ${A.getClass}.")
+ }
+ }
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ gemm(false, false, alpha, A, B, beta, C)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `DenseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: DenseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+ val tAstr = if (!transA) "N" else "T"
+ val tBstr = if (!transB) "N" else "T"
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ nativeBLAS.dgemm(tAstr, tBstr, mA, nB, kA, alpha, A.values, A.numRows, B.values, B.numRows,
+ beta, C.values, C.numRows)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `SparseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: SparseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ val Avals = A.values
+ val Arows = if (!transA) A.rowIndices else A.colPtrs
+ val Acols = if (!transA) A.colPtrs else A.rowIndices
+
+ // Slicing is easy in this case. This is the optimal multiplication setting for sparse matrices
+ if (transA){
+ var colCounterForB = 0
+ if (!transB){ // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var rowCounterForA = 0
+ val Cstart = colCounterForB * mA
+ val Bstart = colCounterForB * kA
+ while (rowCounterForA < mA) {
+ var i = Arows(rowCounterForA)
+ val indEnd = Arows(rowCounterForA + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B.values(Bstart + Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounterForA
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ } else {
+ while (colCounterForB < nB) {
+ var rowCounter = 0
+ val Cstart = colCounterForB * mA
+ while (rowCounter < mA) {
+ var i = Arows(rowCounter)
+ val indEnd = Arows(rowCounter + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B(colCounterForB, Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounter
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounter += 1
+ }
+ colCounterForB += 1
+ }
+ }
+ } else {
+ // Scale matrix first if `beta` is not equal to 0.0
+ if (beta != 0.0){
+ nativeBLAS.dscal(C.values.length, beta, C.values, 1)
+ }
+ // Perform matrix multiplication and add to C. The rows of A are multiplied by the columns of
+ // B, and added to C.
+ var colCounterForB = 0 // the column to be updated in C
+ if (!transB) { // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var colCounterForA = 0 // The column of A to multiply with the row of B
+ while (colCounterForA < kA){
+ var i = Acols(colCounterForA)
+ val indEnd = Acols(colCounterForA + 1)
+ val Bval = B(colCounterForA, colCounterForB)
+ val Cstart = colCounterForB * mA
+ while (i < indEnd){
+ C.values(Cstart + Arows(i)) += Avals(i) * Bval * alpha
+ i += 1
+ }
+ colCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ } else {
+ while (colCounterForB < nB) {
+ var colCounterForA = 0 // The column of A to multiply with the row of B
+ while (colCounterForA < kA){
+ var i = Acols(colCounterForA)
+ val indEnd = Acols(colCounterForA + 1)
+ val Bval = B(colCounterForB, colCounterForA)
+ val Cstart = colCounterForB * mA
+ while (i < indEnd){
+ C.values(Cstart + Arows(i)) += Avals(i) * Bval * alpha
+ i += 1
+ }
+ colCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ }
+ }
+ }
+
+ /**
+ * y := alpha * A * x + beta * y
+ * @param trans specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ * @param beta a scalar that can be used to scale vector y.
+ * @param y the resulting vector y. Size of m x 1.
+ */
+ def gemv(
+ trans: Boolean,
+ alpha: Double,
+ A: Matrix,
+ x: DenseVector,
+ beta: Double,
+ y: DenseVector): Unit = {
+
+ val mA: Int = if (!trans) A.numRows else A.numCols
+ val nx: Int = x.size
+ val nA: Int = if (!trans) A.numCols else A.numRows
+
+ require(nA == nx, s"The columns of A don't match the number of elements of x. A: $nA, x: $nx")
+ require(mA == y.size,
+ s"The rows of A don't match the number of elements of y. A: $mA, y:${y.size}}")
+ if (alpha == 0.0) {
+ logWarning("gemv: alpha is equal to 0. Returning y.")
+ } else {
+ A match {
+ case sparse: SparseMatrix =>
+ gemv(trans, alpha, sparse, x, beta, y)
+ case dense: DenseMatrix =>
+ gemv(trans, alpha, dense, x, beta, y)
+ case _ =>
+ throw new IllegalArgumentException(s"gemv doesn't support matrix type ${A.getClass}.")
+ }
+ }
+ }
+
+ /**
+ * y := alpha * A * x + beta * y
+ *
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ * @param beta a scalar that can be used to scale vector y.
+ * @param y the resulting vector y. Size of m x 1.
+ */
+ def gemv(
+ alpha: Double,
+ A: Matrix,
+ x: DenseVector,
+ beta: Double,
+ y: DenseVector): Unit = {
+ gemv(false, alpha, A, x, beta, y)
+ }
+
+ /**
+ * y := alpha * A * x
+ *
+ * @param trans specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ *
+ * @return `DenseVector` y, the result of the matrix-vector multiplication. Size of m x 1.
+ */
+ def gemv(
+ trans: Boolean,
+ alpha: Double,
+ A: Matrix,
+ x: DenseVector): DenseVector = {
+ val m = if(!trans) A.numRows else A.numCols
+
+ val y: DenseVector = new DenseVector(Array.fill(m)(0.0))
+ gemv(trans, alpha, A, x, 0.0, y)
+
+ y
+ }
+
+ /**
+ * y := alpha * A * x
+ *
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ *
+ * @return `DenseVector` y, the result of the matrix-vector multiplication. Size of m x 1.
+ */
+ def gemv(
+ alpha: Double,
+ A: Matrix,
+ x: DenseVector): DenseVector = {
+ gemv(false, alpha, A, x)
+ }
+
+
+ /**
+ * y := alpha * A * x + beta * y
+ * For `DenseMatrix` A.
+ */
+ private def gemv(
+ trans: Boolean,
+ alpha: Double,
+ A: DenseMatrix,
+ x: DenseVector,
+ beta: Double,
+ y: DenseVector): Unit = {
+ val tStrA = if (!trans) "N" else "T"
+ nativeBLAS.dgemv(tStrA, A.numRows, A.numCols, alpha, A.values, A.numRows, x.values, 1, beta,
--- End diff --
nativeBLAS was faster across the board. Here are the results (in ms):
m | n | f2j | native
---|----|-----|--------
1000 | 500 | 0.7 | 0.1
1000 | 1000 | 2.3 | 0.7
1000 | 5000 | 7.7 | 3.6
5000 | 500 | 3.8 | 2.4
5000 | 1000 | 7.7 | 4.0
5000 | 5000 | 38.4 | 17.4
10000 | 500 | 6.5 | 4.5
10000 | 1000 | 13.0 | 8.9
10000 | 5000 | 75.4 | 36.7
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-56001230
[QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20527/consoleFull) for PR 2294 at commit [`f79db9c`](https://github.com/apache/spark/commit/f79db9c0d82ceea41d59972e3bde9fa2a17b6112).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-54704308
[QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19896/consoleFull) for PR 2294 at commit [`848406c`](https://github.com/apache/spark/commit/848406c1b6779eea9bbcf1dd582b541db46d7dad).
* This patch **fails** unit tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `class SparseMatrix(`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-54703462
[QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19896/consoleFull) for PR 2294 at commit [`848406c`](https://github.com/apache/spark/commit/848406c1b6779eea9bbcf1dd582b541db46d7dad).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17255331
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala ---
@@ -63,7 +88,162 @@ class DenseMatrix(val numRows: Int, val numCols: Int, val values: Array[Double])
override def toArray: Array[Double] = values
- private[mllib] override def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+ private [mllib] def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+
+ private [mllib] def apply(i: Int): Double = values(i)
+
+ private [mllib] def apply(r: Int, c: Int): Double = values(index(r, c))
+
+ private [mllib] def index(r: Int, c: Int): Int = r + numRows * c
+
+ private [mllib] def update(r: Int, c: Int, v: Double){
+ values(index(r, c)) = v
+ }
+
+ def copy = new DenseMatrix(numRows, numCols, values.clone())
+}
+
+/**
+ * Factory methods for [[org.apache.spark.mllib.linalg.DenseMatrix]].
+ *
+ * These methods can be used to generate common matrix types such as the Identity matrix, any
+ * diagonal matrix, zero matrix, and random matrices.
+ */
+object DenseMatrix {
+
+ /**
+ * Generate a `DenseMatrix` consisting of zeros.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values of zeros
+ */
+ def zeros(numRows: Int, numCols: Int) =
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(0.0))
+
+ /**
+ * Generate a `DenseMatrix` consisting of ones.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values of ones
+ */
+ def ones(numRows: Int, numCols: Int) =
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(1.0))
+
+ /**
+ * Generate an Identity Matrix in `DenseMatrix` format.
+ * @param n number of rows and columns of the matrix
+ * @return `DenseMatrix` with size `n` x `n` and values of ones on the diagonal
+ */
+ def eye(n: Int) = {
+ val identity = DenseMatrix.zeros(n,n)
+ for (i <- 0 until n){
+ identity.update(i, i, 1.0)
+ }
+ identity
+ }
+
+ /**
+ * Generate a `DenseMatrix` consisting of i.i.d. uniform random numbers.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values in U(0, 1)
+ */
+ def rand(numRows: Int, numCols: Int) = {
+ val rand = new scala.util.Random
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(rand.nextDouble()))
+ }
+
+ /**
+ * Generate a `DenseMatrix` consisting of i.i.d. gaussian random numbers.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values in N(0, 1)
+ */
+ def randn(numRows: Int, numCols: Int) = {
+ val rand = new scala.util.Random
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(rand.nextGaussian()))
+ }
+
+ /**
+ * Generate a diagonal matrix in `DenseMatrix` format from the supplied values.
+ * @param values values on the diagonal of the matrix
+ * @return Square `DenseMatrix` with size `values.length` x `values.length` and `values`
+ * on the diagonal
+ */
+ def diag(values: Array[Double]) = {
+ val n = values.length
+ val matrix = DenseMatrix.eye(n)
+ for (i <- 0 until n) matrix.update(i, i, values(i))
+ matrix
+ }
+}
+
+/**
+ * Column-majored sparse matrix.
+ * The entry values are stored in Compressed-Column Storage (CCS) format.
--- End diff --
Shall we use `compressed sparse column (CSC)`? Both are correct but `CSC` is more common.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by brkyvz <gi...@git.apache.org>.
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-55982010
Jenkins, test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17255251
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +199,451 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: String,
--- End diff --
Change type to `Boolean`?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17704449
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +201,368 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ if (alpha == 0.0) {
+ logWarning("gemm: alpha is equal to 0. Returning C.")
+ } else {
+ A match {
+ case sparse: SparseMatrix =>
+ gemm(transA, transB, alpha, sparse, B, beta, C)
+ case dense: DenseMatrix =>
+ gemm(transA, transB, alpha, dense, B, beta, C)
+ case _ =>
+ throw new IllegalArgumentException(s"gemm doesn't support matrix type ${A.getClass}.")
+ }
+ }
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ gemm(false, false, alpha, A, B, beta, C)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `DenseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: DenseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+ val tAstr = if (!transA) "N" else "T"
+ val tBstr = if (!transB) "N" else "T"
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ nativeBLAS.dgemm(tAstr, tBstr, mA, nB, kA, alpha, A.values, A.numRows, B.values, B.numRows,
+ beta, C.values, C.numRows)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `SparseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: SparseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ val Avals = A.values
+ val Arows = if (!transA) A.rowIndices else A.colPtrs
+ val Acols = if (!transA) A.colPtrs else A.rowIndices
+
+ // Slicing is easy in this case. This is the optimal multiplication setting for sparse matrices
+ if (transA){
+ var colCounterForB = 0
+ if (!transB){ // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var rowCounterForA = 0
+ val Cstart = colCounterForB * mA
+ val Bstart = colCounterForB * kA
+ while (rowCounterForA < mA) {
+ var i = Arows(rowCounterForA)
+ val indEnd = Arows(rowCounterForA + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B.values(Bstart + Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounterForA
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ } else {
+ while (colCounterForB < nB) {
+ var rowCounter = 0
+ val Cstart = colCounterForB * mA
+ while (rowCounter < mA) {
+ var i = Arows(rowCounter)
+ val indEnd = Arows(rowCounter + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B(colCounterForB, Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounter
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounter += 1
+ }
+ colCounterForB += 1
+ }
+ }
+ } else {
+ // Scale matrix first if `beta` is not equal to 0.0
+ if (beta != 0.0){
+ nativeBLAS.dscal(C.values.length, beta, C.values, 1)
+ }
+ // Perform matrix multiplication and add to C. The rows of A are multiplied by the columns of
+ // B, and added to C.
+ var colCounterForB = 0 // the column to be updated in C
+ if (!transB) { // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var colCounterForA = 0 // The column of A to multiply with the row of B
+ while (colCounterForA < kA){
--- End diff --
add space after `)`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17255341
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala ---
@@ -63,7 +88,162 @@ class DenseMatrix(val numRows: Int, val numCols: Int, val values: Array[Double])
override def toArray: Array[Double] = values
- private[mllib] override def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+ private [mllib] def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+
+ private [mllib] def apply(i: Int): Double = values(i)
+
+ private [mllib] def apply(r: Int, c: Int): Double = values(index(r, c))
+
+ private [mllib] def index(r: Int, c: Int): Int = r + numRows * c
+
+ private [mllib] def update(r: Int, c: Int, v: Double){
+ values(index(r, c)) = v
+ }
+
+ def copy = new DenseMatrix(numRows, numCols, values.clone())
+}
+
+/**
+ * Factory methods for [[org.apache.spark.mllib.linalg.DenseMatrix]].
+ *
+ * These methods can be used to generate common matrix types such as the Identity matrix, any
+ * diagonal matrix, zero matrix, and random matrices.
+ */
+object DenseMatrix {
+
+ /**
+ * Generate a `DenseMatrix` consisting of zeros.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values of zeros
+ */
+ def zeros(numRows: Int, numCols: Int) =
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(0.0))
+
+ /**
+ * Generate a `DenseMatrix` consisting of ones.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values of ones
+ */
+ def ones(numRows: Int, numCols: Int) =
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(1.0))
+
+ /**
+ * Generate an Identity Matrix in `DenseMatrix` format.
+ * @param n number of rows and columns of the matrix
+ * @return `DenseMatrix` with size `n` x `n` and values of ones on the diagonal
+ */
+ def eye(n: Int) = {
+ val identity = DenseMatrix.zeros(n,n)
+ for (i <- 0 until n){
+ identity.update(i, i, 1.0)
+ }
+ identity
+ }
+
+ /**
+ * Generate a `DenseMatrix` consisting of i.i.d. uniform random numbers.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values in U(0, 1)
+ */
+ def rand(numRows: Int, numCols: Int) = {
+ val rand = new scala.util.Random
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(rand.nextDouble()))
+ }
+
+ /**
+ * Generate a `DenseMatrix` consisting of i.i.d. gaussian random numbers.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values in N(0, 1)
+ */
+ def randn(numRows: Int, numCols: Int) = {
+ val rand = new scala.util.Random
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(rand.nextGaussian()))
+ }
+
+ /**
+ * Generate a diagonal matrix in `DenseMatrix` format from the supplied values.
+ * @param values values on the diagonal of the matrix
+ * @return Square `DenseMatrix` with size `values.length` x `values.length` and `values`
+ * on the diagonal
+ */
+ def diag(values: Array[Double]) = {
+ val n = values.length
+ val matrix = DenseMatrix.eye(n)
+ for (i <- 0 until n) matrix.update(i, i, values(i))
+ matrix
+ }
+}
+
+/**
+ * Column-majored sparse matrix.
+ * The entry values are stored in Compressed-Column Storage (CCS) format.
+ * For example, the following matrix
+ * {{{
+ * 1.0 0.0 4.0
+ * 0.0 3.0 5.0
+ * 2.0 0.0 6.0
+ * }}}
+ * is stored as `values: [1.0, 2.0, 3.0, 4.0, 5.0, 6.0]`,
+ * `rowIndices=[0, 2, 1, 0, 1, 2]`, `colPointers=[0, 2, 3, 6]`.
+ *
+ * @param numRows number of rows
+ * @param numCols number of columns
+ * @param colPointers the index corresponding to the start of a new column
+ * @param rowIndices the row index of the entry
+ * @param values non-zero matrix entries in column major
+ */
+class SparseMatrix(
+ val numRows: Int,
+ val numCols: Int,
+ val colPointers: Array[Int],
+ val rowIndices: Array[Int],
+ val values: Array[Double]) extends Matrix {
+
+ require(values.length == rowIndices.length, "The number of row indices and values don't match!")
+ require(colPointers.length == numCols + 1, "The length of the column indices should be the " +
+ s"number of columns + 1. Currently, colPointers.length: ${colPointers.length}, " +
+ s"numCols: $numCols")
+
+ override def toArray: Array[Double] = values
+
+ private [mllib] def toBreeze: BM[Double] =
--- End diff --
remove space between `private` and `[mllib]`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17704446
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +201,368 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ if (alpha == 0.0) {
+ logWarning("gemm: alpha is equal to 0. Returning C.")
+ } else {
+ A match {
+ case sparse: SparseMatrix =>
+ gemm(transA, transB, alpha, sparse, B, beta, C)
+ case dense: DenseMatrix =>
+ gemm(transA, transB, alpha, dense, B, beta, C)
+ case _ =>
+ throw new IllegalArgumentException(s"gemm doesn't support matrix type ${A.getClass}.")
+ }
+ }
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ gemm(false, false, alpha, A, B, beta, C)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `DenseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: DenseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+ val tAstr = if (!transA) "N" else "T"
+ val tBstr = if (!transB) "N" else "T"
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ nativeBLAS.dgemm(tAstr, tBstr, mA, nB, kA, alpha, A.values, A.numRows, B.values, B.numRows,
+ beta, C.values, C.numRows)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `SparseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: SparseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ val Avals = A.values
+ val Arows = if (!transA) A.rowIndices else A.colPtrs
+ val Acols = if (!transA) A.colPtrs else A.rowIndices
+
+ // Slicing is easy in this case. This is the optimal multiplication setting for sparse matrices
+ if (transA){
+ var colCounterForB = 0
+ if (!transB){ // Expensive to put the check inside the loop
--- End diff --
space after `)`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17709072
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala ---
@@ -36,9 +37,42 @@ trait Matrix extends Serializable {
/** Converts to a breeze matrix. */
private[mllib] def toBreeze: BM[Double]
+ /** Gets the i-th element in the array backing the matrix. */
+ private[mllib] def apply(i: Int): Double
--- End diff --
Maybe it is good to remove this method. Because it may cause confusion between DenseMatrix and SparseMatrix.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-54685532
[QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19810/consoleFull) for PR 2294 at commit [`4362ff1`](https://github.com/apache/spark/commit/4362ff1d6b79604b9919da1d550ed466021b8f5c).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17255354
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala ---
@@ -63,7 +88,162 @@ class DenseMatrix(val numRows: Int, val numCols: Int, val values: Array[Double])
override def toArray: Array[Double] = values
- private[mllib] override def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+ private [mllib] def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+
+ private [mllib] def apply(i: Int): Double = values(i)
+
+ private [mllib] def apply(r: Int, c: Int): Double = values(index(r, c))
+
+ private [mllib] def index(r: Int, c: Int): Int = r + numRows * c
+
+ private [mllib] def update(r: Int, c: Int, v: Double){
+ values(index(r, c)) = v
+ }
+
+ def copy = new DenseMatrix(numRows, numCols, values.clone())
+}
+
+/**
+ * Factory methods for [[org.apache.spark.mllib.linalg.DenseMatrix]].
+ *
+ * These methods can be used to generate common matrix types such as the Identity matrix, any
+ * diagonal matrix, zero matrix, and random matrices.
+ */
+object DenseMatrix {
+
+ /**
+ * Generate a `DenseMatrix` consisting of zeros.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values of zeros
+ */
+ def zeros(numRows: Int, numCols: Int) =
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(0.0))
+
+ /**
+ * Generate a `DenseMatrix` consisting of ones.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values of ones
+ */
+ def ones(numRows: Int, numCols: Int) =
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(1.0))
+
+ /**
+ * Generate an Identity Matrix in `DenseMatrix` format.
+ * @param n number of rows and columns of the matrix
+ * @return `DenseMatrix` with size `n` x `n` and values of ones on the diagonal
+ */
+ def eye(n: Int) = {
+ val identity = DenseMatrix.zeros(n,n)
+ for (i <- 0 until n){
+ identity.update(i, i, 1.0)
+ }
+ identity
+ }
+
+ /**
+ * Generate a `DenseMatrix` consisting of i.i.d. uniform random numbers.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values in U(0, 1)
+ */
+ def rand(numRows: Int, numCols: Int) = {
+ val rand = new scala.util.Random
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(rand.nextDouble()))
+ }
+
+ /**
+ * Generate a `DenseMatrix` consisting of i.i.d. gaussian random numbers.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values in N(0, 1)
+ */
+ def randn(numRows: Int, numCols: Int) = {
+ val rand = new scala.util.Random
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(rand.nextGaussian()))
+ }
+
+ /**
+ * Generate a diagonal matrix in `DenseMatrix` format from the supplied values.
+ * @param values values on the diagonal of the matrix
+ * @return Square `DenseMatrix` with size `values.length` x `values.length` and `values`
+ * on the diagonal
+ */
+ def diag(values: Array[Double]) = {
+ val n = values.length
+ val matrix = DenseMatrix.eye(n)
+ for (i <- 0 until n) matrix.update(i, i, values(i))
+ matrix
+ }
+}
+
+/**
+ * Column-majored sparse matrix.
+ * The entry values are stored in Compressed-Column Storage (CCS) format.
+ * For example, the following matrix
+ * {{{
+ * 1.0 0.0 4.0
+ * 0.0 3.0 5.0
+ * 2.0 0.0 6.0
+ * }}}
+ * is stored as `values: [1.0, 2.0, 3.0, 4.0, 5.0, 6.0]`,
+ * `rowIndices=[0, 2, 1, 0, 1, 2]`, `colPointers=[0, 2, 3, 6]`.
+ *
+ * @param numRows number of rows
+ * @param numCols number of columns
+ * @param colPointers the index corresponding to the start of a new column
+ * @param rowIndices the row index of the entry
+ * @param values non-zero matrix entries in column major
+ */
+class SparseMatrix(
+ val numRows: Int,
+ val numCols: Int,
+ val colPointers: Array[Int],
+ val rowIndices: Array[Int],
+ val values: Array[Double]) extends Matrix {
+
+ require(values.length == rowIndices.length, "The number of row indices and values don't match!")
+ require(colPointers.length == numCols + 1, "The length of the column indices should be the " +
+ s"number of columns + 1. Currently, colPointers.length: ${colPointers.length}, " +
+ s"numCols: $numCols")
+
+ override def toArray: Array[Double] = values
+
+ private [mllib] def toBreeze: BM[Double] =
+ new BSM[Double](values, numRows, numCols, colPointers, rowIndices)
+
+ private [mllib] def apply(i: Int): Double = values(i)
+
+ private [mllib] def apply(r: Int, c: Int): Double = {
+ val ind = index(r,c)
+ if (ind == -1) 0.0 else values(ind)
+ }
+
+ private [mllib] def index(r: Int, c: Int): Int = {
+ val regionStart = colPointers(c)
+ val regionEnd = colPointers(c + 1)
+ val region = rowIndices.slice(regionStart, regionEnd)
+ if (region.contains(r)){
+ region.indexOf(r) + regionStart
+ } else {
+ -1 // return zero element
+ }
+ }
+
+ // TODO(Burak): Maybe convert to Breeze to update zero entries? I can't think of any MLlib
+ // TODO: algorithm that would use mutable Sparse Matrices
+ private [mllib] def update(r: Int, c: Int, v: Double){
+ val ind = index(r, c)
+ if (ind == -1){
+ throw new IllegalArgumentException("The given row and column indices correspond to a zero " +
+ "value. Sparse Matrices are currently immutable.")
--- End diff --
The error message should be more precise. We can update an existing entry but cannot insert a new entry.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17255300
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala ---
@@ -36,9 +36,34 @@ trait Matrix extends Serializable {
/** Converts to a breeze matrix. */
private[mllib] def toBreeze: BM[Double]
- /** Gets the (i, j)-th element. */
- private[mllib] def apply(i: Int, j: Int): Double = toBreeze(i, j)
+ /** Gets the i-th element in the array backing the matrix. */
+ private[mllib] def apply(i: Int): Double
+ /** Gets the (r, c)-th element. */
+ private[mllib] def apply(r: Int, c: Int): Double
--- End diff --
`i, j` is more common
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17278811
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +199,369 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ A match {
+ case sparse: SparseMatrix =>
+ gemm(transA, transB, alpha, sparse, B, beta, C)
+ case dense: DenseMatrix =>
+ gemm(transA, transB, alpha, dense, B, beta, C)
+ case _ =>
+ throw new IllegalArgumentException(s"gemm doesn't support matrix type ${A.getClass}.")
+ }
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
+ gemm(false, false, alpha, A, B, beta, C)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `DenseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: DenseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+ val tAstr = if (!transA) "N" else "T"
+ val tBstr = if (!transB) "N" else "T"
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ nativeBLAS.dgemm(tAstr, tBstr, mA, nB, kA, alpha, A.values, A.numRows, B.values, B.numRows,
+ beta, C.values, C.numRows)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `SparseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: SparseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ val Avals = A.values
+ val Arows = if (!transA) A.rowIndices else A.colPtrs
+ val Acols = if (!transA) A.colPtrs else A.rowIndices
+
+ // Slicing is easy in this case. This is the optimal multiplication setting for sparse matrices
+ if (transA){
+ var colCounterForB = 0
+ if (!transB){ // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var rowCounterForA = 0
+ val Cstart = colCounterForB * mA
+ while (rowCounterForA < mA) {
+ var i = Arows(rowCounterForA)
+ val indEnd = Arows(rowCounterForA + 1)
+ val Bstart = colCounterForB * kA
--- End diff --
move this line to the outer loop (after line 319)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17704453
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +201,368 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ if (alpha == 0.0) {
+ logWarning("gemm: alpha is equal to 0. Returning C.")
+ } else {
+ A match {
+ case sparse: SparseMatrix =>
+ gemm(transA, transB, alpha, sparse, B, beta, C)
+ case dense: DenseMatrix =>
+ gemm(transA, transB, alpha, dense, B, beta, C)
+ case _ =>
+ throw new IllegalArgumentException(s"gemm doesn't support matrix type ${A.getClass}.")
+ }
+ }
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ gemm(false, false, alpha, A, B, beta, C)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `DenseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: DenseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+ val tAstr = if (!transA) "N" else "T"
+ val tBstr = if (!transB) "N" else "T"
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ nativeBLAS.dgemm(tAstr, tBstr, mA, nB, kA, alpha, A.values, A.numRows, B.values, B.numRows,
+ beta, C.values, C.numRows)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `SparseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: SparseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ val Avals = A.values
+ val Arows = if (!transA) A.rowIndices else A.colPtrs
+ val Acols = if (!transA) A.colPtrs else A.rowIndices
+
+ // Slicing is easy in this case. This is the optimal multiplication setting for sparse matrices
+ if (transA){
+ var colCounterForB = 0
+ if (!transB){ // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var rowCounterForA = 0
+ val Cstart = colCounterForB * mA
+ val Bstart = colCounterForB * kA
+ while (rowCounterForA < mA) {
+ var i = Arows(rowCounterForA)
+ val indEnd = Arows(rowCounterForA + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B.values(Bstart + Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounterForA
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ } else {
+ while (colCounterForB < nB) {
+ var rowCounter = 0
+ val Cstart = colCounterForB * mA
+ while (rowCounter < mA) {
+ var i = Arows(rowCounter)
+ val indEnd = Arows(rowCounter + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B(colCounterForB, Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounter
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounter += 1
+ }
+ colCounterForB += 1
+ }
+ }
+ } else {
+ // Scale matrix first if `beta` is not equal to 0.0
+ if (beta != 0.0){
+ nativeBLAS.dscal(C.values.length, beta, C.values, 1)
+ }
+ // Perform matrix multiplication and add to C. The rows of A are multiplied by the columns of
+ // B, and added to C.
+ var colCounterForB = 0 // the column to be updated in C
+ if (!transB) { // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var colCounterForA = 0 // The column of A to multiply with the row of B
+ while (colCounterForA < kA){
+ var i = Acols(colCounterForA)
+ val indEnd = Acols(colCounterForA + 1)
+ val Bval = B(colCounterForA, colCounterForB)
+ val Cstart = colCounterForB * mA
+ while (i < indEnd){
+ C.values(Cstart + Arows(i)) += Avals(i) * Bval * alpha
--- End diff --
compute `Bval * alpha` in the outer loop
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17704450
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +201,368 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ if (alpha == 0.0) {
+ logWarning("gemm: alpha is equal to 0. Returning C.")
+ } else {
+ A match {
+ case sparse: SparseMatrix =>
+ gemm(transA, transB, alpha, sparse, B, beta, C)
+ case dense: DenseMatrix =>
+ gemm(transA, transB, alpha, dense, B, beta, C)
+ case _ =>
+ throw new IllegalArgumentException(s"gemm doesn't support matrix type ${A.getClass}.")
+ }
+ }
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ gemm(false, false, alpha, A, B, beta, C)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `DenseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: DenseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+ val tAstr = if (!transA) "N" else "T"
+ val tBstr = if (!transB) "N" else "T"
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ nativeBLAS.dgemm(tAstr, tBstr, mA, nB, kA, alpha, A.values, A.numRows, B.values, B.numRows,
+ beta, C.values, C.numRows)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `SparseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: SparseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ val Avals = A.values
+ val Arows = if (!transA) A.rowIndices else A.colPtrs
+ val Acols = if (!transA) A.colPtrs else A.rowIndices
+
+ // Slicing is easy in this case. This is the optimal multiplication setting for sparse matrices
+ if (transA){
+ var colCounterForB = 0
+ if (!transB){ // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var rowCounterForA = 0
+ val Cstart = colCounterForB * mA
+ val Bstart = colCounterForB * kA
+ while (rowCounterForA < mA) {
+ var i = Arows(rowCounterForA)
+ val indEnd = Arows(rowCounterForA + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B.values(Bstart + Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounterForA
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ } else {
+ while (colCounterForB < nB) {
+ var rowCounter = 0
+ val Cstart = colCounterForB * mA
+ while (rowCounter < mA) {
+ var i = Arows(rowCounter)
+ val indEnd = Arows(rowCounter + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B(colCounterForB, Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounter
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounter += 1
+ }
+ colCounterForB += 1
+ }
+ }
+ } else {
+ // Scale matrix first if `beta` is not equal to 0.0
+ if (beta != 0.0){
+ nativeBLAS.dscal(C.values.length, beta, C.values, 1)
+ }
+ // Perform matrix multiplication and add to C. The rows of A are multiplied by the columns of
+ // B, and added to C.
+ var colCounterForB = 0 // the column to be updated in C
+ if (!transB) { // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var colCounterForA = 0 // The column of A to multiply with the row of B
+ while (colCounterForA < kA){
+ var i = Acols(colCounterForA)
+ val indEnd = Acols(colCounterForA + 1)
+ val Bval = B(colCounterForA, colCounterForB)
--- End diff --
We can cache the start index of column `colCounterForB`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17255293
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +199,451 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: String,
+ transB: String,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
+ var mA: Int = A.numRows
+ var nB: Int = B.numCols
+ var kA: Int = A.numCols
+ var kB: Int = B.numRows
+
+ if (transA == "T" || transA=="t"){
+ mA = A.numCols
+ kA = A.numRows
+ }
+ require(transA == "T" || transA == "t" || transA == "N" || transA == "n",
+ s"""Invalid argument used for transA: $transA. Must be \"N\", \"n\", \"T\", or \"t\"""")
+ if (transB == "T" || transB=="t"){
+ nB = B.numRows
+ kB = B.numCols
+ }
+ require(transB == "T" || transB == "t" || transB == "N" || transB == "n",
+ s"""Invalid argument used for transB: $transB. Must be \"N\", \"n\", \"T\", or \"t\"""")
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ A match {
+ case sparse: SparseMatrix =>
+ gemm(transA, transB, alpha, sparse, B, beta, C)
+ case dense: DenseMatrix =>
+ gemm(transA, transB, alpha, dense, B, beta, C)
+ case _ =>
+ throw new IllegalArgumentException(s"gemm doesn't support matrix type ${A.getClass}.")
+ }
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
+ gemm("N", "N", alpha, A, B, beta, C)
+ }
+
+ /**
+ * C := alpha * A * B
+ *
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ *
+ * @return The resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: String,
+ transB: String,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix) : DenseMatrix = {
+ var mA: Int = A.numRows
+ var nB: Int = B.numCols
+ var kA: Int = A.numCols
+ var kB: Int = B.numRows
+
+ if (transA == "T" || transA=="T"){
+ mA = A.numCols
+ kA = A.numRows
+ }
+ if (transB == "T" || transB=="T"){
+ nB = B.numRows
+ kB = B.numCols
+ }
+
+ val C: DenseMatrix = DenseMatrix.zeros(mA, nB)
+ gemm(transA, transB, alpha, A, B, 0.0, C)
+
+ C
+ }
+
+ /**
+ * C := alpha * A * B
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ *
+ * @return The resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix) : DenseMatrix = {
+ gemm("N", "N", alpha, A, B)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `DenseMatrix` A.
+ */
+ private def gemm(
+ transA: String,
+ transB: String,
+ alpha: Double,
+ A: DenseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
+ var mA: Int = A.numRows
+ var nB: Int = B.numCols
+ var kA: Int = A.numCols
+ var kB: Int = B.numRows
+
+ if (transA == "T" || transA=="t"){
+ mA = A.numCols
+ kA = A.numRows
+ }
+ if (transB == "T" || transB=="t"){
+ nB = B.numRows
+ kB = B.numCols
+ }
+
+ nativeBLAS.dgemm(transA,transB, mA, nB, kA, alpha, A.toArray, A.numRows, B.toArray, B.numRows,
+ beta, C.toArray, C.numRows)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `SparseMatrix` A.
+ */
+ private def gemm(
+ transA: String,
+ transB: String,
+ alpha: Double,
+ A: SparseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
+ var transposeA = false
+ var transposeB = false
+ var mA: Int = A.numRows
+ var nB: Int = B.numCols
+ var kA: Int = A.numCols
+
+ if (transA == "T" || transA=="t"){
+ mA = A.numCols
+ kA = A.numRows
+ transposeA = true
+ }
+ if (transB == "T" || transB=="t"){
+ nB = B.numRows
+ transposeB = true
+ }
+ val Avals = A.toArray
+ val Arows = if (!transposeA) A.rowIndices else A.colPointers
+ val Acols = if (!transposeA) A.colPointers else A.rowIndices
+
+ // Slicing is easy in this case. This is the optimal multiplication setting for sparse matrices
+ if (transposeA){
+ var colCounter = 0
+ while (colCounter < nB){ // Tests showed that the outer loop being columns was faster
+ var rowCounter = 0
+ while (rowCounter < mA){
+ val indStart = Arows(rowCounter)
+ val indEnd = Arows(rowCounter + 1)
+ var elementCount = 0 // Loop over non-zero entries in column (actually the row indices,
+ // since this is a transposed multiplication
+ var sum = 0.0
+ while(indStart + elementCount < indEnd){
+ val AcolIndex = Acols(indStart + elementCount)
+ val Bval = if (!transposeB) B(AcolIndex, colCounter) else B(colCounter, AcolIndex)
+ sum += Avals(indStart + elementCount) * Bval
+ elementCount += 1
+ }
+ C.update(rowCounter, colCounter, beta * C(rowCounter, colCounter) + sum)
+ rowCounter += 1
+ }
+ colCounter += 1
+ }
+ } else {
+
+ // Scale matrix first if `beta` is not equal to 0.0
+ if (beta != 0.0){
+ var i = 0
+ val Clength = C.numCols * C.numRows
+ while ( i < Clength) {
+ C.values(i) *= beta
+ i += 1
+ }
+ }
+
+ // Perform matrix multiplication and add to C. The rows of A are multiplied by the columns of
+ // B, and added to C.
+ var colCounterForA = 0 // The column of A to multiply with the row of B
+ while (colCounterForA < kA){
+ val indStart = Acols(colCounterForA)
+ val indEnd = Acols(colCounterForA + 1)
+
+ var elementCount = 0 // Loop over non-zero entries in column
+ while (indStart + elementCount < indEnd){
+ var colCounterForB = 0 // the column to be updated in C
+ val rowIndex = Arows(indStart + elementCount) // the row to be updated in C
+ while (colCounterForB < nB){
+ val Bval =
+ if (!transposeB){
--- End diff --
ditto: make it an outer block
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17278837
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +199,369 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ A match {
+ case sparse: SparseMatrix =>
+ gemm(transA, transB, alpha, sparse, B, beta, C)
+ case dense: DenseMatrix =>
+ gemm(transA, transB, alpha, dense, B, beta, C)
+ case _ =>
+ throw new IllegalArgumentException(s"gemm doesn't support matrix type ${A.getClass}.")
+ }
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
+ gemm(false, false, alpha, A, B, beta, C)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `DenseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: DenseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+ val tAstr = if (!transA) "N" else "T"
+ val tBstr = if (!transB) "N" else "T"
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ nativeBLAS.dgemm(tAstr, tBstr, mA, nB, kA, alpha, A.values, A.numRows, B.values, B.numRows,
+ beta, C.values, C.numRows)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `SparseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: SparseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ val Avals = A.values
+ val Arows = if (!transA) A.rowIndices else A.colPtrs
+ val Acols = if (!transA) A.colPtrs else A.rowIndices
+
+ // Slicing is easy in this case. This is the optimal multiplication setting for sparse matrices
+ if (transA){
+ var colCounterForB = 0
+ if (!transB){ // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var rowCounterForA = 0
+ val Cstart = colCounterForB * mA
+ while (rowCounterForA < mA) {
+ var i = Arows(rowCounterForA)
+ val indEnd = Arows(rowCounterForA + 1)
+ val Bstart = colCounterForB * kA
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B(Bstart + Acols(i))
+ i += 1
+ }
+ C.values(rowCounterForA + Cstart) = beta * C.values(rowCounterForA + Cstart) + sum
--- End diff --
cache `rowCounterForA + Cstart`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17709089
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala ---
@@ -83,6 +219,24 @@ object Matrices {
}
/**
+ * Creates a column-majored sparse matrix in Compressed Sparse Column (CSC) format.
+ *
+ * @param numRows number of rows
+ * @param numCols number of columns
+ * @param colPointers the index corresponding to the start of a new column
+ * @param rowIndices the row index of the entry
+ * @param values non-zero matrix entries in column major
+ */
+ def sparse(
+ numRows: Int,
+ numCols: Int,
+ colPointers: Array[Int],
--- End diff --
`colPtrs`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-55684623
test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-54850135
[QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19982/consoleFull) for PR 2294 at commit [`848406c`](https://github.com/apache/spark/commit/848406c1b6779eea9bbcf1dd582b541db46d7dad).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17255285
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +199,451 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: String,
+ transB: String,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
+ var mA: Int = A.numRows
+ var nB: Int = B.numCols
+ var kA: Int = A.numCols
+ var kB: Int = B.numRows
+
+ if (transA == "T" || transA=="t"){
+ mA = A.numCols
+ kA = A.numRows
+ }
+ require(transA == "T" || transA == "t" || transA == "N" || transA == "n",
+ s"""Invalid argument used for transA: $transA. Must be \"N\", \"n\", \"T\", or \"t\"""")
+ if (transB == "T" || transB=="t"){
+ nB = B.numRows
+ kB = B.numCols
+ }
+ require(transB == "T" || transB == "t" || transB == "N" || transB == "n",
+ s"""Invalid argument used for transB: $transB. Must be \"N\", \"n\", \"T\", or \"t\"""")
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ A match {
+ case sparse: SparseMatrix =>
+ gemm(transA, transB, alpha, sparse, B, beta, C)
+ case dense: DenseMatrix =>
+ gemm(transA, transB, alpha, dense, B, beta, C)
+ case _ =>
+ throw new IllegalArgumentException(s"gemm doesn't support matrix type ${A.getClass}.")
+ }
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
+ gemm("N", "N", alpha, A, B, beta, C)
+ }
+
+ /**
+ * C := alpha * A * B
+ *
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ *
+ * @return The resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: String,
+ transB: String,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix) : DenseMatrix = {
+ var mA: Int = A.numRows
+ var nB: Int = B.numCols
+ var kA: Int = A.numCols
+ var kB: Int = B.numRows
+
+ if (transA == "T" || transA=="T"){
+ mA = A.numCols
+ kA = A.numRows
+ }
+ if (transB == "T" || transB=="T"){
+ nB = B.numRows
+ kB = B.numCols
+ }
+
+ val C: DenseMatrix = DenseMatrix.zeros(mA, nB)
+ gemm(transA, transB, alpha, A, B, 0.0, C)
+
+ C
+ }
+
+ /**
+ * C := alpha * A * B
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ *
+ * @return The resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix) : DenseMatrix = {
+ gemm("N", "N", alpha, A, B)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `DenseMatrix` A.
+ */
+ private def gemm(
+ transA: String,
+ transB: String,
+ alpha: Double,
+ A: DenseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
+ var mA: Int = A.numRows
+ var nB: Int = B.numCols
+ var kA: Int = A.numCols
+ var kB: Int = B.numRows
+
+ if (transA == "T" || transA=="t"){
+ mA = A.numCols
+ kA = A.numRows
+ }
+ if (transB == "T" || transB=="t"){
+ nB = B.numRows
+ kB = B.numCols
+ }
+
+ nativeBLAS.dgemm(transA,transB, mA, nB, kA, alpha, A.toArray, A.numRows, B.toArray, B.numRows,
+ beta, C.toArray, C.numRows)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `SparseMatrix` A.
+ */
+ private def gemm(
+ transA: String,
+ transB: String,
+ alpha: Double,
+ A: SparseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
+ var transposeA = false
+ var transposeB = false
+ var mA: Int = A.numRows
+ var nB: Int = B.numCols
+ var kA: Int = A.numCols
+
+ if (transA == "T" || transA=="t"){
+ mA = A.numCols
+ kA = A.numRows
+ transposeA = true
+ }
+ if (transB == "T" || transB=="t"){
+ nB = B.numRows
+ transposeB = true
+ }
+ val Avals = A.toArray
+ val Arows = if (!transposeA) A.rowIndices else A.colPointers
+ val Acols = if (!transposeA) A.colPointers else A.rowIndices
+
+ // Slicing is easy in this case. This is the optimal multiplication setting for sparse matrices
+ if (transposeA){
+ var colCounter = 0
+ while (colCounter < nB){ // Tests showed that the outer loop being columns was faster
+ var rowCounter = 0
+ while (rowCounter < mA){
+ val indStart = Arows(rowCounter)
+ val indEnd = Arows(rowCounter + 1)
+ var elementCount = 0 // Loop over non-zero entries in column (actually the row indices,
+ // since this is a transposed multiplication
+ var sum = 0.0
+ while(indStart + elementCount < indEnd){
+ val AcolIndex = Acols(indStart + elementCount)
+ val Bval = if (!transposeB) B(AcolIndex, colCounter) else B(colCounter, AcolIndex)
--- End diff --
do not put the if check in the inner loop, which is expensive. make four outer blocks instead.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-54940399
[QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20031/consoleFull) for PR 2294 at commit [`a85ccb7`](https://github.com/apache/spark/commit/a85ccb712d83d20b178b40c32bd473d7d018a88f).
* This patch **fails** unit tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `class SparseMatrix(`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17709077
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala ---
@@ -36,9 +37,42 @@ trait Matrix extends Serializable {
/** Converts to a breeze matrix. */
private[mllib] def toBreeze: BM[Double]
+ /** Gets the i-th element in the array backing the matrix. */
+ private[mllib] def apply(i: Int): Double
+
/** Gets the (i, j)-th element. */
- private[mllib] def apply(i: Int, j: Int): Double = toBreeze(i, j)
+ private[mllib] def apply(i: Int, j: Int): Double
+
+ /** Return the index for the (i, j)-th element in the backing array. */
+ private[mllib] def index(i: Int, j: Int): Int
+
+ /** Update element at (i, j) */
+ private[mllib] def update(i: Int, j: Int, v: Double)
+
+ /** Get a deep copy of the matrix. */
+ def copy: Matrix
+
+ /** Convenience method for `Matrix`-`DenseMatrix` multiplication. */
+ def times(y: DenseMatrix): DenseMatrix = {
+ val C: DenseMatrix = Matrices.zeros(numRows, y.numCols).asInstanceOf[DenseMatrix]
+ BLAS.gemm(false, false, 1.0, this, y, 0.0, C)
+ C
+ }
+
+ /** Convenience method for `Matrix`-`DenseVector` multiplication. */
+ def times(y: DenseVector): DenseVector = BLAS.gemv(1.0, this, y)
+
+ /** Convenience method for `Matrix`^T^-`DenseMatrix` multiplication. */
+ def transposeTimes(y: DenseMatrix): DenseMatrix = {
--- End diff --
another approach is to have a boolean flag in matrix indicating whether it is transposed or not.
~~~
A.trans().times(y)
~~~
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17255278
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +199,451 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: String,
+ transB: String,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
+ var mA: Int = A.numRows
+ var nB: Int = B.numCols
+ var kA: Int = A.numCols
+ var kB: Int = B.numRows
+
+ if (transA == "T" || transA=="t"){
+ mA = A.numCols
+ kA = A.numRows
+ }
+ require(transA == "T" || transA == "t" || transA == "N" || transA == "n",
+ s"""Invalid argument used for transA: $transA. Must be \"N\", \"n\", \"T\", or \"t\"""")
+ if (transB == "T" || transB=="t"){
+ nB = B.numRows
+ kB = B.numCols
+ }
+ require(transB == "T" || transB == "t" || transB == "N" || transB == "n",
+ s"""Invalid argument used for transB: $transB. Must be \"N\", \"n\", \"T\", or \"t\"""")
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ A match {
+ case sparse: SparseMatrix =>
+ gemm(transA, transB, alpha, sparse, B, beta, C)
+ case dense: DenseMatrix =>
+ gemm(transA, transB, alpha, dense, B, beta, C)
+ case _ =>
+ throw new IllegalArgumentException(s"gemm doesn't support matrix type ${A.getClass}.")
+ }
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
+ gemm("N", "N", alpha, A, B, beta, C)
+ }
+
+ /**
+ * C := alpha * A * B
+ *
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ *
+ * @return The resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: String,
+ transB: String,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix) : DenseMatrix = {
+ var mA: Int = A.numRows
+ var nB: Int = B.numCols
+ var kA: Int = A.numCols
+ var kB: Int = B.numRows
+
+ if (transA == "T" || transA=="T"){
+ mA = A.numCols
+ kA = A.numRows
+ }
+ if (transB == "T" || transB=="T"){
+ nB = B.numRows
+ kB = B.numCols
+ }
+
+ val C: DenseMatrix = DenseMatrix.zeros(mA, nB)
+ gemm(transA, transB, alpha, A, B, 0.0, C)
+
+ C
+ }
+
+ /**
+ * C := alpha * A * B
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ *
+ * @return The resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix) : DenseMatrix = {
+ gemm("N", "N", alpha, A, B)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `DenseMatrix` A.
+ */
+ private def gemm(
+ transA: String,
+ transB: String,
+ alpha: Double,
+ A: DenseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
+ var mA: Int = A.numRows
+ var nB: Int = B.numCols
+ var kA: Int = A.numCols
+ var kB: Int = B.numRows
+
+ if (transA == "T" || transA=="t"){
+ mA = A.numCols
+ kA = A.numRows
+ }
+ if (transB == "T" || transB=="t"){
+ nB = B.numRows
+ kB = B.numCols
+ }
+
+ nativeBLAS.dgemm(transA,transB, mA, nB, kA, alpha, A.toArray, A.numRows, B.toArray, B.numRows,
--- End diff --
add space after `,`
replace `toArray` by `values`. The name `toArray` suggests conversion.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17255307
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala ---
@@ -63,7 +88,162 @@ class DenseMatrix(val numRows: Int, val numCols: Int, val values: Array[Double])
override def toArray: Array[Double] = values
- private[mllib] override def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+ private [mllib] def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+
+ private [mllib] def apply(i: Int): Double = values(i)
+
+ private [mllib] def apply(r: Int, c: Int): Double = values(index(r, c))
+
+ private [mllib] def index(r: Int, c: Int): Int = r + numRows * c
+
+ private [mllib] def update(r: Int, c: Int, v: Double){
+ values(index(r, c)) = v
+ }
+
+ def copy = new DenseMatrix(numRows, numCols, values.clone())
+}
+
+/**
+ * Factory methods for [[org.apache.spark.mllib.linalg.DenseMatrix]].
+ *
+ * These methods can be used to generate common matrix types such as the Identity matrix, any
+ * diagonal matrix, zero matrix, and random matrices.
+ */
+object DenseMatrix {
--- End diff --
We can move those functions to `Matrices`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17709085
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala ---
@@ -59,11 +93,113 @@ trait Matrix extends Serializable {
*/
class DenseMatrix(val numRows: Int, val numCols: Int, val values: Array[Double]) extends Matrix {
- require(values.length == numRows * numCols)
+ require(values.length == numRows * numCols, "The number of values supplied doesn't match the " +
+ s"size of the matrix! values.length: ${values.length}, numRows * numCols: ${numRows * numCols}")
override def toArray: Array[Double] = values
- private[mllib] override def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+ private[mllib] def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+
+ private[mllib] def apply(i: Int): Double = values(i)
+
+ private[mllib] def apply(i: Int, j: Int): Double = values(index(i, j))
+
+ private[mllib] def index(i: Int, j: Int): Int = i + numRows * j
+
+ private[mllib] def update(i: Int, j: Int, v: Double){
+ values(index(i, j)) = v
+ }
+
+ override def copy = new DenseMatrix(numRows, numCols, values.clone())
+}
+
+/**
+ * Column-majored sparse matrix.
+ * The entry values are stored in Compressed Sparse Column (CSC) format.
+ * For example, the following matrix
+ * {{{
+ * 1.0 0.0 4.0
+ * 0.0 3.0 5.0
+ * 2.0 0.0 6.0
+ * }}}
+ * is stored as `values: [1.0, 2.0, 3.0, 4.0, 5.0, 6.0]`,
+ * `rowIndices=[0, 2, 1, 0, 1, 2]`, `colPointers=[0, 2, 3, 6]`.
+ *
+ * @param numRows number of rows
+ * @param numCols number of columns
+ * @param colPtrs the index corresponding to the start of a new column
+ * @param rowIndices the row index of the entry
+ * @param values non-zero matrix entries in column major
+ */
+class SparseMatrix(
+ val numRows: Int,
+ val numCols: Int,
+ val colPtrs: Array[Int],
+ val rowIndices: Array[Int],
+ val values: Array[Double]) extends Matrix {
+
+ require(values.length == rowIndices.length, "The number of row indices and values don't match! " +
+ s"values.length: ${values.length}, rowIndices.length: ${rowIndices.length}")
+ require(colPtrs.length == numCols + 1, "The length of the column indices should be the " +
+ s"number of columns + 1. Currently, colPointers.length: ${colPtrs.length}, " +
+ s"numCols: $numCols")
+
+ override def toArray: Array[Double] = {
+ val arr = Array.fill(numRows * numCols)(0.0)
+ var j = 0
+ while (j < numCols) {
+ var i = colPtrs(j)
+ val indEnd = colPtrs(j + 1)
+ val offset = j * numRows
+ while (i < indEnd) {
+ val rowIndex = rowIndices(i)
+ arr(offset + rowIndex) = values(i)
+ i += 1
+ }
+ j += 1
+ }
+ arr
+ }
+
+ private[mllib] def toBreeze: BM[Double] =
+ new BSM[Double](values, numRows, numCols, colPtrs, rowIndices)
+
+ private[mllib] def apply(i: Int): Double = values(i)
--- End diff --
ditto: remove this method and use `.values(i)` directly
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by brkyvz <gi...@git.apache.org>.
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-56116462
@ScrapCodes, I have some mima incompatibility issues here. I've added a few methods to the trait Matrix in MLlib, and that causes a MissingMethodProblem. I sealed the trait, and added the exclude statements to the MimaExcludes, but the tests still fail.
`dev/mima` passes on my local machine.
Do you have any idea of what might be going on? The commits d162684 and
272feb9 were to address this issue.
Can you please take a look?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17709092
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala ---
@@ -93,9 +247,84 @@ object Matrices {
require(dm.majorStride == dm.rows,
"Do not support stride size different from the number of rows.")
new DenseMatrix(dm.rows, dm.cols, dm.data)
+ case sm: BSM[Double] =>
+ new SparseMatrix(sm.rows, sm.cols, sm.colPtrs, sm.rowIndices, sm.data)
case _ =>
throw new UnsupportedOperationException(
s"Do not support conversion from type ${breeze.getClass.getName}.")
}
}
+
+ /**
+ * Generate a `DenseMatrix` consisting of zeros.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values of zeros
+ */
+ def zeros(numRows: Int, numCols: Int): Matrix =
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(0.0))
--- End diff --
`new Array` instead of `Array.fill`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-55833289
[QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20432/consoleFull) for PR 2294 at commit [`d510c8f`](https://github.com/apache/spark/commit/d510c8f940faee3bdb2b00306f150bc99630396b).
* This patch **fails** unit tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `class SparseMatrix(`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-55383849
[QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20225/consoleFull) for PR 2294 at commit [`a85ccb7`](https://github.com/apache/spark/commit/a85ccb712d83d20b178b40c32bd473d7d018a88f).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17709102
--- Diff: mllib/src/test/scala/org/apache/spark/mllib/linalg/MatricesSuite.scala ---
@@ -36,4 +36,79 @@ class MatricesSuite extends FunSuite {
Matrices.dense(3, 2, Array(0.0, 1.0, 2.0))
}
}
+
+ test("sparse matrix construction") {
+ val m = 3
+ val n = 2
+ val values = Array(1.0, 2.0, 4.0, 5.0)
+ val colIndices = Array(0, 2, 4)
+ val rowIndices = Array(1, 2, 1, 2)
+ val mat = Matrices.sparse(m, n, colIndices, rowIndices, values).asInstanceOf[SparseMatrix]
+ assert(mat.numRows === m)
+ assert(mat.numCols === n)
+ assert(mat.values.eq(values), "should not copy data")
+ }
+
+ test("sparse matrix construction with wrong number of elements") {
+ intercept[RuntimeException] {
+ Matrices.sparse(3, 2, Array(0, 1), Array(1, 2, 1), Array(0.0, 1.0, 2.0))
+ }
+
+ intercept[RuntimeException] {
+ Matrices.sparse(3, 2, Array(0, 1, 2), Array(1, 2), Array(0.0, 1.0, 2.0))
+ }
+ }
+
+ test("matrix copies are deep copies") {
+ val m = 3
+ val n = 2
+
+ val denseMat = Matrices.dense(m, n, Array(0.0, 1.0, 2.0, 3.0, 4.0, 5.0))
+ val denseCopy = denseMat.copy
+
+ assert(!denseMat.toArray.eq(denseCopy.toArray))
+
+ val values = Array(1.0, 2.0, 4.0, 5.0)
+ val colIndices = Array(0, 2, 4)
--- End diff --
`colPtrs`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17278884
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +199,369 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ A match {
+ case sparse: SparseMatrix =>
+ gemm(transA, transB, alpha, sparse, B, beta, C)
+ case dense: DenseMatrix =>
+ gemm(transA, transB, alpha, dense, B, beta, C)
+ case _ =>
+ throw new IllegalArgumentException(s"gemm doesn't support matrix type ${A.getClass}.")
+ }
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
+ gemm(false, false, alpha, A, B, beta, C)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `DenseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: DenseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+ val tAstr = if (!transA) "N" else "T"
+ val tBstr = if (!transB) "N" else "T"
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ nativeBLAS.dgemm(tAstr, tBstr, mA, nB, kA, alpha, A.values, A.numRows, B.values, B.numRows,
+ beta, C.values, C.numRows)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `SparseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: SparseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ val Avals = A.values
+ val Arows = if (!transA) A.rowIndices else A.colPtrs
+ val Acols = if (!transA) A.colPtrs else A.rowIndices
+
+ // Slicing is easy in this case. This is the optimal multiplication setting for sparse matrices
+ if (transA){
+ var colCounterForB = 0
+ if (!transB){ // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var rowCounterForA = 0
+ val Cstart = colCounterForB * mA
+ while (rowCounterForA < mA) {
+ var i = Arows(rowCounterForA)
+ val indEnd = Arows(rowCounterForA + 1)
+ val Bstart = colCounterForB * kA
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B(Bstart + Acols(i))
+ i += 1
+ }
+ C.values(rowCounterForA + Cstart) = beta * C.values(rowCounterForA + Cstart) + sum
+ rowCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ } else {
+ while (colCounterForB < nB) {
+ var rowCounter = 0
+ val Cstart = colCounterForB * mA
+ while (rowCounter < mA) {
+ var i = Arows(rowCounter)
+ val indEnd = Arows(rowCounter + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B(colCounterForB, Acols(i))
+ i += 1
+ }
+ C.values(rowCounter + Cstart) = beta * C.values(rowCounter + Cstart) + sum
+ rowCounter += 1
+ }
+ colCounterForB += 1
+ }
+ }
+ } else {
+ // Scale matrix first if `beta` is not equal to 0.0
+ if (beta != 0.0){
+ var i = 0
+ val Clength = C.numCols * C.numRows
+ while ( i < Clength) {
+ C.values(i) *= beta
+ i += 1
+ }
+ }
+ // Perform matrix multiplication and add to C. The rows of A are multiplied by the columns of
+ // B, and added to C.
+ var colCounterForB = 0 // the column to be updated in C
+ if (!transB) { // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var colCounterForA = 0 // The column of A to multiply with the row of B
+ while (colCounterForA < kA){
+ var i = Acols(colCounterForA)
+ val indEnd = Acols(colCounterForA + 1)
+ val Bval = B(colCounterForA, colCounterForB)
+ val Cstart = colCounterForB * mA
+ while (i < indEnd){
+ C.values(Arows(i) + Cstart) += Avals(i) * Bval
--- End diff --
`Cstart + Arows(i)` (use the same ordering across implementation)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17255333
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala ---
@@ -63,7 +88,162 @@ class DenseMatrix(val numRows: Int, val numCols: Int, val values: Array[Double])
override def toArray: Array[Double] = values
- private[mllib] override def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+ private [mllib] def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+
+ private [mllib] def apply(i: Int): Double = values(i)
+
+ private [mllib] def apply(r: Int, c: Int): Double = values(index(r, c))
+
+ private [mllib] def index(r: Int, c: Int): Int = r + numRows * c
+
+ private [mllib] def update(r: Int, c: Int, v: Double){
+ values(index(r, c)) = v
+ }
+
+ def copy = new DenseMatrix(numRows, numCols, values.clone())
+}
+
+/**
+ * Factory methods for [[org.apache.spark.mllib.linalg.DenseMatrix]].
+ *
+ * These methods can be used to generate common matrix types such as the Identity matrix, any
+ * diagonal matrix, zero matrix, and random matrices.
+ */
+object DenseMatrix {
+
+ /**
+ * Generate a `DenseMatrix` consisting of zeros.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values of zeros
+ */
+ def zeros(numRows: Int, numCols: Int) =
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(0.0))
+
+ /**
+ * Generate a `DenseMatrix` consisting of ones.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values of ones
+ */
+ def ones(numRows: Int, numCols: Int) =
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(1.0))
+
+ /**
+ * Generate an Identity Matrix in `DenseMatrix` format.
+ * @param n number of rows and columns of the matrix
+ * @return `DenseMatrix` with size `n` x `n` and values of ones on the diagonal
+ */
+ def eye(n: Int) = {
+ val identity = DenseMatrix.zeros(n,n)
+ for (i <- 0 until n){
+ identity.update(i, i, 1.0)
+ }
+ identity
+ }
+
+ /**
+ * Generate a `DenseMatrix` consisting of i.i.d. uniform random numbers.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values in U(0, 1)
+ */
+ def rand(numRows: Int, numCols: Int) = {
+ val rand = new scala.util.Random
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(rand.nextDouble()))
+ }
+
+ /**
+ * Generate a `DenseMatrix` consisting of i.i.d. gaussian random numbers.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values in N(0, 1)
+ */
+ def randn(numRows: Int, numCols: Int) = {
+ val rand = new scala.util.Random
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(rand.nextGaussian()))
+ }
+
+ /**
+ * Generate a diagonal matrix in `DenseMatrix` format from the supplied values.
+ * @param values values on the diagonal of the matrix
+ * @return Square `DenseMatrix` with size `values.length` x `values.length` and `values`
+ * on the diagonal
+ */
+ def diag(values: Array[Double]) = {
+ val n = values.length
+ val matrix = DenseMatrix.eye(n)
+ for (i <- 0 until n) matrix.update(i, i, values(i))
+ matrix
+ }
+}
+
+/**
+ * Column-majored sparse matrix.
+ * The entry values are stored in Compressed-Column Storage (CCS) format.
+ * For example, the following matrix
+ * {{{
+ * 1.0 0.0 4.0
+ * 0.0 3.0 5.0
+ * 2.0 0.0 6.0
+ * }}}
+ * is stored as `values: [1.0, 2.0, 3.0, 4.0, 5.0, 6.0]`,
+ * `rowIndices=[0, 2, 1, 0, 1, 2]`, `colPointers=[0, 2, 3, 6]`.
+ *
+ * @param numRows number of rows
+ * @param numCols number of columns
+ * @param colPointers the index corresponding to the start of a new column
+ * @param rowIndices the row index of the entry
+ * @param values non-zero matrix entries in column major
+ */
+class SparseMatrix(
+ val numRows: Int,
+ val numCols: Int,
+ val colPointers: Array[Int],
+ val rowIndices: Array[Int],
+ val values: Array[Double]) extends Matrix {
+
+ require(values.length == rowIndices.length, "The number of row indices and values don't match!")
--- End diff --
put `values.length` and `rowIndices.length` to the error message
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by brkyvz <gi...@git.apache.org>.
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-56123894
@ScrapCodes THANKS A LOT! That fixed it! I didn't realize I didn't update my local repo for such a long time.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17255336
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala ---
@@ -63,7 +88,162 @@ class DenseMatrix(val numRows: Int, val numCols: Int, val values: Array[Double])
override def toArray: Array[Double] = values
- private[mllib] override def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+ private [mllib] def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+
+ private [mllib] def apply(i: Int): Double = values(i)
+
+ private [mllib] def apply(r: Int, c: Int): Double = values(index(r, c))
+
+ private [mllib] def index(r: Int, c: Int): Int = r + numRows * c
+
+ private [mllib] def update(r: Int, c: Int, v: Double){
+ values(index(r, c)) = v
+ }
+
+ def copy = new DenseMatrix(numRows, numCols, values.clone())
+}
+
+/**
+ * Factory methods for [[org.apache.spark.mllib.linalg.DenseMatrix]].
+ *
+ * These methods can be used to generate common matrix types such as the Identity matrix, any
+ * diagonal matrix, zero matrix, and random matrices.
+ */
+object DenseMatrix {
+
+ /**
+ * Generate a `DenseMatrix` consisting of zeros.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values of zeros
+ */
+ def zeros(numRows: Int, numCols: Int) =
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(0.0))
+
+ /**
+ * Generate a `DenseMatrix` consisting of ones.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values of ones
+ */
+ def ones(numRows: Int, numCols: Int) =
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(1.0))
+
+ /**
+ * Generate an Identity Matrix in `DenseMatrix` format.
+ * @param n number of rows and columns of the matrix
+ * @return `DenseMatrix` with size `n` x `n` and values of ones on the diagonal
+ */
+ def eye(n: Int) = {
+ val identity = DenseMatrix.zeros(n,n)
+ for (i <- 0 until n){
+ identity.update(i, i, 1.0)
+ }
+ identity
+ }
+
+ /**
+ * Generate a `DenseMatrix` consisting of i.i.d. uniform random numbers.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values in U(0, 1)
+ */
+ def rand(numRows: Int, numCols: Int) = {
+ val rand = new scala.util.Random
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(rand.nextDouble()))
+ }
+
+ /**
+ * Generate a `DenseMatrix` consisting of i.i.d. gaussian random numbers.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values in N(0, 1)
+ */
+ def randn(numRows: Int, numCols: Int) = {
+ val rand = new scala.util.Random
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(rand.nextGaussian()))
+ }
+
+ /**
+ * Generate a diagonal matrix in `DenseMatrix` format from the supplied values.
+ * @param values values on the diagonal of the matrix
+ * @return Square `DenseMatrix` with size `values.length` x `values.length` and `values`
+ * on the diagonal
+ */
+ def diag(values: Array[Double]) = {
+ val n = values.length
+ val matrix = DenseMatrix.eye(n)
+ for (i <- 0 until n) matrix.update(i, i, values(i))
+ matrix
+ }
+}
+
+/**
+ * Column-majored sparse matrix.
+ * The entry values are stored in Compressed-Column Storage (CCS) format.
+ * For example, the following matrix
+ * {{{
+ * 1.0 0.0 4.0
+ * 0.0 3.0 5.0
+ * 2.0 0.0 6.0
+ * }}}
+ * is stored as `values: [1.0, 2.0, 3.0, 4.0, 5.0, 6.0]`,
+ * `rowIndices=[0, 2, 1, 0, 1, 2]`, `colPointers=[0, 2, 3, 6]`.
+ *
+ * @param numRows number of rows
+ * @param numCols number of columns
+ * @param colPointers the index corresponding to the start of a new column
+ * @param rowIndices the row index of the entry
+ * @param values non-zero matrix entries in column major
+ */
+class SparseMatrix(
+ val numRows: Int,
+ val numCols: Int,
+ val colPointers: Array[Int],
+ val rowIndices: Array[Int],
+ val values: Array[Double]) extends Matrix {
+
+ require(values.length == rowIndices.length, "The number of row indices and values don't match!")
+ require(colPointers.length == numCols + 1, "The length of the column indices should be the " +
+ s"number of columns + 1. Currently, colPointers.length: ${colPointers.length}, " +
+ s"numCols: $numCols")
+
+ override def toArray: Array[Double] = values
--- End diff --
This is wrong. `toArray` should return the dense representation stored in a column-majored flat array.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17255253
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +199,451 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: String,
+ transB: String,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
--- End diff --
add `: Unit = ` to be explicit
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17255265
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +199,451 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: String,
+ transB: String,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
+ var mA: Int = A.numRows
+ var nB: Int = B.numCols
+ var kA: Int = A.numCols
+ var kB: Int = B.numRows
+
+ if (transA == "T" || transA=="t"){
+ mA = A.numCols
+ kA = A.numRows
+ }
+ require(transA == "T" || transA == "t" || transA == "N" || transA == "n",
+ s"""Invalid argument used for transA: $transA. Must be \"N\", \"n\", \"T\", or \"t\"""")
+ if (transB == "T" || transB=="t"){
+ nB = B.numRows
+ kB = B.numCols
+ }
+ require(transB == "T" || transB == "t" || transB == "N" || transB == "n",
+ s"""Invalid argument used for transB: $transB. Must be \"N\", \"n\", \"T\", or \"t\"""")
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ A match {
+ case sparse: SparseMatrix =>
+ gemm(transA, transB, alpha, sparse, B, beta, C)
+ case dense: DenseMatrix =>
+ gemm(transA, transB, alpha, dense, B, beta, C)
+ case _ =>
+ throw new IllegalArgumentException(s"gemm doesn't support matrix type ${A.getClass}.")
+ }
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
+ gemm("N", "N", alpha, A, B, beta, C)
+ }
+
+ /**
+ * C := alpha * A * B
+ *
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ *
+ * @return The resulting matrix C. Size of m x n.
+ */
+ def gemm(
--- End diff --
Shall we remove this method? Developers should be aware of temporary memory allocation.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17255317
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala ---
@@ -63,7 +88,162 @@ class DenseMatrix(val numRows: Int, val numCols: Int, val values: Array[Double])
override def toArray: Array[Double] = values
- private[mllib] override def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+ private [mllib] def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+
+ private [mllib] def apply(i: Int): Double = values(i)
+
+ private [mllib] def apply(r: Int, c: Int): Double = values(index(r, c))
+
+ private [mllib] def index(r: Int, c: Int): Int = r + numRows * c
+
+ private [mllib] def update(r: Int, c: Int, v: Double){
+ values(index(r, c)) = v
+ }
+
+ def copy = new DenseMatrix(numRows, numCols, values.clone())
+}
+
+/**
+ * Factory methods for [[org.apache.spark.mllib.linalg.DenseMatrix]].
+ *
+ * These methods can be used to generate common matrix types such as the Identity matrix, any
+ * diagonal matrix, zero matrix, and random matrices.
+ */
+object DenseMatrix {
+
+ /**
+ * Generate a `DenseMatrix` consisting of zeros.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values of zeros
+ */
+ def zeros(numRows: Int, numCols: Int) =
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(0.0))
+
+ /**
+ * Generate a `DenseMatrix` consisting of ones.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values of ones
+ */
+ def ones(numRows: Int, numCols: Int) =
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(1.0))
+
+ /**
+ * Generate an Identity Matrix in `DenseMatrix` format.
+ * @param n number of rows and columns of the matrix
+ * @return `DenseMatrix` with size `n` x `n` and values of ones on the diagonal
+ */
+ def eye(n: Int) = {
+ val identity = DenseMatrix.zeros(n,n)
+ for (i <- 0 until n){
+ identity.update(i, i, 1.0)
+ }
+ identity
+ }
+
+ /**
+ * Generate a `DenseMatrix` consisting of i.i.d. uniform random numbers.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values in U(0, 1)
+ */
+ def rand(numRows: Int, numCols: Int) = {
+ val rand = new scala.util.Random
--- End diff --
use `XORShiftRandom` for speed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17255287
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +199,451 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: String,
+ transB: String,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
+ var mA: Int = A.numRows
+ var nB: Int = B.numCols
+ var kA: Int = A.numCols
+ var kB: Int = B.numRows
+
+ if (transA == "T" || transA=="t"){
+ mA = A.numCols
+ kA = A.numRows
+ }
+ require(transA == "T" || transA == "t" || transA == "N" || transA == "n",
+ s"""Invalid argument used for transA: $transA. Must be \"N\", \"n\", \"T\", or \"t\"""")
+ if (transB == "T" || transB=="t"){
+ nB = B.numRows
+ kB = B.numCols
+ }
+ require(transB == "T" || transB == "t" || transB == "N" || transB == "n",
+ s"""Invalid argument used for transB: $transB. Must be \"N\", \"n\", \"T\", or \"t\"""")
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ A match {
+ case sparse: SparseMatrix =>
+ gemm(transA, transB, alpha, sparse, B, beta, C)
+ case dense: DenseMatrix =>
+ gemm(transA, transB, alpha, dense, B, beta, C)
+ case _ =>
+ throw new IllegalArgumentException(s"gemm doesn't support matrix type ${A.getClass}.")
+ }
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
+ gemm("N", "N", alpha, A, B, beta, C)
+ }
+
+ /**
+ * C := alpha * A * B
+ *
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ *
+ * @return The resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: String,
+ transB: String,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix) : DenseMatrix = {
+ var mA: Int = A.numRows
+ var nB: Int = B.numCols
+ var kA: Int = A.numCols
+ var kB: Int = B.numRows
+
+ if (transA == "T" || transA=="T"){
+ mA = A.numCols
+ kA = A.numRows
+ }
+ if (transB == "T" || transB=="T"){
+ nB = B.numRows
+ kB = B.numCols
+ }
+
+ val C: DenseMatrix = DenseMatrix.zeros(mA, nB)
+ gemm(transA, transB, alpha, A, B, 0.0, C)
+
+ C
+ }
+
+ /**
+ * C := alpha * A * B
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ *
+ * @return The resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix) : DenseMatrix = {
+ gemm("N", "N", alpha, A, B)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `DenseMatrix` A.
+ */
+ private def gemm(
+ transA: String,
+ transB: String,
+ alpha: Double,
+ A: DenseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
+ var mA: Int = A.numRows
+ var nB: Int = B.numCols
+ var kA: Int = A.numCols
+ var kB: Int = B.numRows
+
+ if (transA == "T" || transA=="t"){
+ mA = A.numCols
+ kA = A.numRows
+ }
+ if (transB == "T" || transB=="t"){
+ nB = B.numRows
+ kB = B.numCols
+ }
+
+ nativeBLAS.dgemm(transA,transB, mA, nB, kA, alpha, A.toArray, A.numRows, B.toArray, B.numRows,
+ beta, C.toArray, C.numRows)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `SparseMatrix` A.
+ */
+ private def gemm(
+ transA: String,
+ transB: String,
+ alpha: Double,
+ A: SparseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
+ var transposeA = false
+ var transposeB = false
+ var mA: Int = A.numRows
+ var nB: Int = B.numCols
+ var kA: Int = A.numCols
+
+ if (transA == "T" || transA=="t"){
+ mA = A.numCols
+ kA = A.numRows
+ transposeA = true
+ }
+ if (transB == "T" || transB=="t"){
+ nB = B.numRows
+ transposeB = true
+ }
+ val Avals = A.toArray
+ val Arows = if (!transposeA) A.rowIndices else A.colPointers
+ val Acols = if (!transposeA) A.colPointers else A.rowIndices
+
+ // Slicing is easy in this case. This is the optimal multiplication setting for sparse matrices
+ if (transposeA){
+ var colCounter = 0
+ while (colCounter < nB){ // Tests showed that the outer loop being columns was faster
+ var rowCounter = 0
+ while (rowCounter < mA){
+ val indStart = Arows(rowCounter)
+ val indEnd = Arows(rowCounter + 1)
+ var elementCount = 0 // Loop over non-zero entries in column (actually the row indices,
+ // since this is a transposed multiplication
+ var sum = 0.0
+ while(indStart + elementCount < indEnd){
+ val AcolIndex = Acols(indStart + elementCount)
+ val Bval = if (!transposeB) B(AcolIndex, colCounter) else B(colCounter, AcolIndex)
+ sum += Avals(indStart + elementCount) * Bval
+ elementCount += 1
+ }
+ C.update(rowCounter, colCounter, beta * C(rowCounter, colCounter) + sum)
--- End diff --
update directly on `C.values`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17255254
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +199,451 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: String,
+ transB: String,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
+ var mA: Int = A.numRows
--- End diff --
make it a `val`:
~~~
val mA = if (transA) A.numCols else A.numRows
~~~
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17255272
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +199,451 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: String,
+ transB: String,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
+ var mA: Int = A.numRows
+ var nB: Int = B.numCols
+ var kA: Int = A.numCols
+ var kB: Int = B.numRows
+
+ if (transA == "T" || transA=="t"){
+ mA = A.numCols
+ kA = A.numRows
+ }
+ require(transA == "T" || transA == "t" || transA == "N" || transA == "n",
+ s"""Invalid argument used for transA: $transA. Must be \"N\", \"n\", \"T\", or \"t\"""")
+ if (transB == "T" || transB=="t"){
+ nB = B.numRows
+ kB = B.numCols
+ }
+ require(transB == "T" || transB == "t" || transB == "N" || transB == "n",
+ s"""Invalid argument used for transB: $transB. Must be \"N\", \"n\", \"T\", or \"t\"""")
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ A match {
+ case sparse: SparseMatrix =>
+ gemm(transA, transB, alpha, sparse, B, beta, C)
+ case dense: DenseMatrix =>
+ gemm(transA, transB, alpha, dense, B, beta, C)
+ case _ =>
+ throw new IllegalArgumentException(s"gemm doesn't support matrix type ${A.getClass}.")
+ }
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix) {
+ gemm("N", "N", alpha, A, B, beta, C)
+ }
+
+ /**
+ * C := alpha * A * B
+ *
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ *
+ * @return The resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: String,
+ transB: String,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix) : DenseMatrix = {
+ var mA: Int = A.numRows
+ var nB: Int = B.numCols
+ var kA: Int = A.numCols
+ var kB: Int = B.numRows
+
+ if (transA == "T" || transA=="T"){
+ mA = A.numCols
+ kA = A.numRows
+ }
+ if (transB == "T" || transB=="T"){
+ nB = B.numRows
+ kB = B.numCols
+ }
+
+ val C: DenseMatrix = DenseMatrix.zeros(mA, nB)
+ gemm(transA, transB, alpha, A, B, 0.0, C)
+
+ C
+ }
+
+ /**
+ * C := alpha * A * B
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ *
+ * @return The resulting matrix C. Size of m x n.
+ */
+ def gemm(
--- End diff --
ditto. Remove this one too.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-55977479
Adding new methods to a trait is a break change. We can mark `Vector` and `Matrix` as sealed, so no one can extend them. From Jenkins log:
~~~
[error] * method transposeTimes(org.apache.spark.mllib.linalg.DenseVector)org.apache.spark.mllib.linalg.DenseVector in trait org.apache.spark.mllib.linalg.Matrix does not have a correspondent in old version
[error] filter with: ProblemFilters.exclude[MissingMethodProblem]("org.apache.spark.mllib.linalg.Matrix.transposeTimes")
[error] * method transposeTimes(org.apache.spark.mllib.linalg.DenseMatrix)org.apache.spark.mllib.linalg.DenseMatrix in trait org.apache.spark.mllib.linalg.Matrix does not have a correspondent in old version
[error] filter with: ProblemFilters.exclude[MissingMethodProblem]("org.apache.spark.mllib.linalg.Matrix.transposeTimes")
[error] * method times(org.apache.spark.mllib.linalg.DenseVector)org.apache.spark.mllib.linalg.DenseVector in trait org.apache.spark.mllib.linalg.Matrix does not have a correspondent in old version
[error] filter with: ProblemFilters.exclude[MissingMethodProblem]("org.apache.spark.mllib.linalg.Matrix.times")
[error] * method times(org.apache.spark.mllib.linalg.DenseMatrix)org.apache.spark.mllib.linalg.DenseMatrix in trait org.apache.spark.mllib.linalg.Matrix does not have a correspondent in old version
[error] filter with: ProblemFilters.exclude[MissingMethodProblem]("org.apache.spark.mllib.linalg.Matrix.times")
[error] * method copy()org.apache.spark.mllib.linalg.Matrix in trait org.apache.spark.mllib.linalg.Matrix does not have a correspondent in old version
[error] filter with: ProblemFilters.exclude[MissingMethodProblem]("org.apache.spark.mllib.linalg.Matrix.copy")
~~~
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17709079
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala ---
@@ -59,11 +93,113 @@ trait Matrix extends Serializable {
*/
class DenseMatrix(val numRows: Int, val numCols: Int, val values: Array[Double]) extends Matrix {
- require(values.length == numRows * numCols)
+ require(values.length == numRows * numCols, "The number of values supplied doesn't match the " +
+ s"size of the matrix! values.length: ${values.length}, numRows * numCols: ${numRows * numCols}")
override def toArray: Array[Double] = values
- private[mllib] override def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+ private[mllib] def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+
+ private[mllib] def apply(i: Int): Double = values(i)
+
+ private[mllib] def apply(i: Int, j: Int): Double = values(index(i, j))
+
+ private[mllib] def index(i: Int, j: Int): Int = i + numRows * j
+
+ private[mllib] def update(i: Int, j: Int, v: Double){
--- End diff --
`: Unit = {`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17709060
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +201,368 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ if (alpha == 0.0) {
+ logWarning("gemm: alpha is equal to 0. Returning C.")
+ } else {
+ A match {
+ case sparse: SparseMatrix =>
+ gemm(transA, transB, alpha, sparse, B, beta, C)
+ case dense: DenseMatrix =>
+ gemm(transA, transB, alpha, dense, B, beta, C)
+ case _ =>
+ throw new IllegalArgumentException(s"gemm doesn't support matrix type ${A.getClass}.")
+ }
+ }
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ gemm(false, false, alpha, A, B, beta, C)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `DenseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: DenseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+ val tAstr = if (!transA) "N" else "T"
+ val tBstr = if (!transB) "N" else "T"
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ nativeBLAS.dgemm(tAstr, tBstr, mA, nB, kA, alpha, A.values, A.numRows, B.values, B.numRows,
+ beta, C.values, C.numRows)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `SparseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: SparseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ val Avals = A.values
+ val Arows = if (!transA) A.rowIndices else A.colPtrs
+ val Acols = if (!transA) A.colPtrs else A.rowIndices
+
+ // Slicing is easy in this case. This is the optimal multiplication setting for sparse matrices
+ if (transA){
+ var colCounterForB = 0
+ if (!transB){ // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var rowCounterForA = 0
+ val Cstart = colCounterForB * mA
+ val Bstart = colCounterForB * kA
+ while (rowCounterForA < mA) {
+ var i = Arows(rowCounterForA)
+ val indEnd = Arows(rowCounterForA + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B.values(Bstart + Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounterForA
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ } else {
+ while (colCounterForB < nB) {
+ var rowCounter = 0
+ val Cstart = colCounterForB * mA
+ while (rowCounter < mA) {
+ var i = Arows(rowCounter)
+ val indEnd = Arows(rowCounter + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B(colCounterForB, Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounter
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounter += 1
+ }
+ colCounterForB += 1
+ }
+ }
+ } else {
+ // Scale matrix first if `beta` is not equal to 0.0
+ if (beta != 0.0){
+ nativeBLAS.dscal(C.values.length, beta, C.values, 1)
+ }
+ // Perform matrix multiplication and add to C. The rows of A are multiplied by the columns of
+ // B, and added to C.
+ var colCounterForB = 0 // the column to be updated in C
+ if (!transB) { // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var colCounterForA = 0 // The column of A to multiply with the row of B
+ while (colCounterForA < kA){
+ var i = Acols(colCounterForA)
+ val indEnd = Acols(colCounterForA + 1)
+ val Bval = B(colCounterForA, colCounterForB)
+ val Cstart = colCounterForB * mA
+ while (i < indEnd){
+ C.values(Cstart + Arows(i)) += Avals(i) * Bval * alpha
+ i += 1
+ }
+ colCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ } else {
+ while (colCounterForB < nB) {
+ var colCounterForA = 0 // The column of A to multiply with the row of B
+ while (colCounterForA < kA){
+ var i = Acols(colCounterForA)
+ val indEnd = Acols(colCounterForA + 1)
+ val Bval = B(colCounterForB, colCounterForA)
+ val Cstart = colCounterForB * mA
+ while (i < indEnd){
+ C.values(Cstart + Arows(i)) += Avals(i) * Bval * alpha
+ i += 1
+ }
+ colCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ }
+ }
+ }
+
+ /**
+ * y := alpha * A * x + beta * y
+ * @param trans specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ * @param beta a scalar that can be used to scale vector y.
+ * @param y the resulting vector y. Size of m x 1.
+ */
+ def gemv(
+ trans: Boolean,
+ alpha: Double,
+ A: Matrix,
+ x: DenseVector,
+ beta: Double,
+ y: DenseVector): Unit = {
+
+ val mA: Int = if (!trans) A.numRows else A.numCols
+ val nx: Int = x.size
+ val nA: Int = if (!trans) A.numCols else A.numRows
+
+ require(nA == nx, s"The columns of A don't match the number of elements of x. A: $nA, x: $nx")
+ require(mA == y.size,
+ s"The rows of A don't match the number of elements of y. A: $mA, y:${y.size}}")
+ if (alpha == 0.0) {
+ logWarning("gemv: alpha is equal to 0. Returning y.")
+ } else {
+ A match {
+ case sparse: SparseMatrix =>
+ gemv(trans, alpha, sparse, x, beta, y)
+ case dense: DenseMatrix =>
+ gemv(trans, alpha, dense, x, beta, y)
+ case _ =>
+ throw new IllegalArgumentException(s"gemv doesn't support matrix type ${A.getClass}.")
+ }
+ }
+ }
+
+ /**
+ * y := alpha * A * x + beta * y
+ *
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ * @param beta a scalar that can be used to scale vector y.
+ * @param y the resulting vector y. Size of m x 1.
+ */
+ def gemv(
+ alpha: Double,
+ A: Matrix,
+ x: DenseVector,
+ beta: Double,
+ y: DenseVector): Unit = {
+ gemv(false, alpha, A, x, beta, y)
+ }
+
+ /**
+ * y := alpha * A * x
+ *
+ * @param trans specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ *
+ * @return `DenseVector` y, the result of the matrix-vector multiplication. Size of m x 1.
+ */
+ def gemv(
+ trans: Boolean,
+ alpha: Double,
+ A: Matrix,
+ x: DenseVector): DenseVector = {
+ val m = if(!trans) A.numRows else A.numCols
+
+ val y: DenseVector = new DenseVector(Array.fill(m)(0.0))
--- End diff --
`new Array[Double](m)`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17704444
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +201,368 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
--- End diff --
Doc needs update. It is a boolean now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by brkyvz <gi...@git.apache.org>.
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-55827933
Jenkins, test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17709065
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +201,368 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ if (alpha == 0.0) {
+ logWarning("gemm: alpha is equal to 0. Returning C.")
+ } else {
+ A match {
+ case sparse: SparseMatrix =>
+ gemm(transA, transB, alpha, sparse, B, beta, C)
+ case dense: DenseMatrix =>
+ gemm(transA, transB, alpha, dense, B, beta, C)
+ case _ =>
+ throw new IllegalArgumentException(s"gemm doesn't support matrix type ${A.getClass}.")
+ }
+ }
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ gemm(false, false, alpha, A, B, beta, C)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `DenseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: DenseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+ val tAstr = if (!transA) "N" else "T"
+ val tBstr = if (!transB) "N" else "T"
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ nativeBLAS.dgemm(tAstr, tBstr, mA, nB, kA, alpha, A.values, A.numRows, B.values, B.numRows,
+ beta, C.values, C.numRows)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `SparseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: SparseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ val Avals = A.values
+ val Arows = if (!transA) A.rowIndices else A.colPtrs
+ val Acols = if (!transA) A.colPtrs else A.rowIndices
+
+ // Slicing is easy in this case. This is the optimal multiplication setting for sparse matrices
+ if (transA){
+ var colCounterForB = 0
+ if (!transB){ // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var rowCounterForA = 0
+ val Cstart = colCounterForB * mA
+ val Bstart = colCounterForB * kA
+ while (rowCounterForA < mA) {
+ var i = Arows(rowCounterForA)
+ val indEnd = Arows(rowCounterForA + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B.values(Bstart + Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounterForA
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ } else {
+ while (colCounterForB < nB) {
+ var rowCounter = 0
+ val Cstart = colCounterForB * mA
+ while (rowCounter < mA) {
+ var i = Arows(rowCounter)
+ val indEnd = Arows(rowCounter + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B(colCounterForB, Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounter
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounter += 1
+ }
+ colCounterForB += 1
+ }
+ }
+ } else {
+ // Scale matrix first if `beta` is not equal to 0.0
+ if (beta != 0.0){
+ nativeBLAS.dscal(C.values.length, beta, C.values, 1)
+ }
+ // Perform matrix multiplication and add to C. The rows of A are multiplied by the columns of
+ // B, and added to C.
+ var colCounterForB = 0 // the column to be updated in C
+ if (!transB) { // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var colCounterForA = 0 // The column of A to multiply with the row of B
+ while (colCounterForA < kA){
+ var i = Acols(colCounterForA)
+ val indEnd = Acols(colCounterForA + 1)
+ val Bval = B(colCounterForA, colCounterForB)
+ val Cstart = colCounterForB * mA
+ while (i < indEnd){
+ C.values(Cstart + Arows(i)) += Avals(i) * Bval * alpha
+ i += 1
+ }
+ colCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ } else {
+ while (colCounterForB < nB) {
+ var colCounterForA = 0 // The column of A to multiply with the row of B
+ while (colCounterForA < kA){
+ var i = Acols(colCounterForA)
+ val indEnd = Acols(colCounterForA + 1)
+ val Bval = B(colCounterForB, colCounterForA)
+ val Cstart = colCounterForB * mA
+ while (i < indEnd){
+ C.values(Cstart + Arows(i)) += Avals(i) * Bval * alpha
+ i += 1
+ }
+ colCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ }
+ }
+ }
+
+ /**
+ * y := alpha * A * x + beta * y
+ * @param trans specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ * @param beta a scalar that can be used to scale vector y.
+ * @param y the resulting vector y. Size of m x 1.
+ */
+ def gemv(
+ trans: Boolean,
+ alpha: Double,
+ A: Matrix,
+ x: DenseVector,
+ beta: Double,
+ y: DenseVector): Unit = {
+
+ val mA: Int = if (!trans) A.numRows else A.numCols
+ val nx: Int = x.size
+ val nA: Int = if (!trans) A.numCols else A.numRows
+
+ require(nA == nx, s"The columns of A don't match the number of elements of x. A: $nA, x: $nx")
+ require(mA == y.size,
+ s"The rows of A don't match the number of elements of y. A: $mA, y:${y.size}}")
+ if (alpha == 0.0) {
+ logWarning("gemv: alpha is equal to 0. Returning y.")
+ } else {
+ A match {
+ case sparse: SparseMatrix =>
+ gemv(trans, alpha, sparse, x, beta, y)
+ case dense: DenseMatrix =>
+ gemv(trans, alpha, dense, x, beta, y)
+ case _ =>
+ throw new IllegalArgumentException(s"gemv doesn't support matrix type ${A.getClass}.")
+ }
+ }
+ }
+
+ /**
+ * y := alpha * A * x + beta * y
+ *
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ * @param beta a scalar that can be used to scale vector y.
+ * @param y the resulting vector y. Size of m x 1.
+ */
+ def gemv(
+ alpha: Double,
+ A: Matrix,
+ x: DenseVector,
+ beta: Double,
+ y: DenseVector): Unit = {
+ gemv(false, alpha, A, x, beta, y)
+ }
+
+ /**
+ * y := alpha * A * x
+ *
+ * @param trans specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ *
+ * @return `DenseVector` y, the result of the matrix-vector multiplication. Size of m x 1.
+ */
+ def gemv(
+ trans: Boolean,
+ alpha: Double,
+ A: Matrix,
+ x: DenseVector): DenseVector = {
+ val m = if(!trans) A.numRows else A.numCols
+
+ val y: DenseVector = new DenseVector(Array.fill(m)(0.0))
+ gemv(trans, alpha, A, x, 0.0, y)
+
+ y
+ }
+
+ /**
+ * y := alpha * A * x
+ *
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ *
+ * @return `DenseVector` y, the result of the matrix-vector multiplication. Size of m x 1.
+ */
+ def gemv(
+ alpha: Double,
+ A: Matrix,
+ x: DenseVector): DenseVector = {
+ gemv(false, alpha, A, x)
+ }
+
+
+ /**
+ * y := alpha * A * x + beta * y
+ * For `DenseMatrix` A.
+ */
+ private def gemv(
+ trans: Boolean,
+ alpha: Double,
+ A: DenseMatrix,
+ x: DenseVector,
+ beta: Double,
+ y: DenseVector): Unit = {
+ val tStrA = if (!trans) "N" else "T"
+ nativeBLAS.dgemv(tStrA, A.numRows, A.numCols, alpha, A.values, A.numRows, x.values, 1, beta,
--- End diff --
It is worth testing whether we should use `f2jBLAS` or `nativeBLAS` for level-2.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-54696840
[QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19815/consoleFull) for PR 2294 at commit [`8dcb763`](https://github.com/apache/spark/commit/8dcb7632093fc2d87a626b85e08ddabcbf119543).
* This patch **fails** unit tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `class SparseMatrix(val numRows: Int,`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by brkyvz <gi...@git.apache.org>.
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-54932214
Performance results on commit "eeb13eb"
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-55982351
[QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20506/consoleFull) for PR 2294 at commit [`418def8`](https://github.com/apache/spark/commit/418def8e940b93f0d24e9b3158ecc0e130d16a83).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-56123782
[QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20562/consoleFull) for PR 2294 at commit [`88814ed`](https://github.com/apache/spark/commit/88814ed215e81bb6b945c8fff60bdb822ac2cb96).
* This patch **passes** unit tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `sealed trait Matrix extends Serializable `
* `class SparseMatrix(`
* `sealed trait Vector extends Serializable `
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17704452
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +201,368 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ if (alpha == 0.0) {
+ logWarning("gemm: alpha is equal to 0. Returning C.")
+ } else {
+ A match {
+ case sparse: SparseMatrix =>
+ gemm(transA, transB, alpha, sparse, B, beta, C)
+ case dense: DenseMatrix =>
+ gemm(transA, transB, alpha, dense, B, beta, C)
+ case _ =>
+ throw new IllegalArgumentException(s"gemm doesn't support matrix type ${A.getClass}.")
+ }
+ }
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ gemm(false, false, alpha, A, B, beta, C)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `DenseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: DenseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+ val tAstr = if (!transA) "N" else "T"
+ val tBstr = if (!transB) "N" else "T"
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ nativeBLAS.dgemm(tAstr, tBstr, mA, nB, kA, alpha, A.values, A.numRows, B.values, B.numRows,
+ beta, C.values, C.numRows)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `SparseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: SparseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ val Avals = A.values
+ val Arows = if (!transA) A.rowIndices else A.colPtrs
+ val Acols = if (!transA) A.colPtrs else A.rowIndices
+
+ // Slicing is easy in this case. This is the optimal multiplication setting for sparse matrices
+ if (transA){
+ var colCounterForB = 0
+ if (!transB){ // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var rowCounterForA = 0
+ val Cstart = colCounterForB * mA
+ val Bstart = colCounterForB * kA
+ while (rowCounterForA < mA) {
+ var i = Arows(rowCounterForA)
+ val indEnd = Arows(rowCounterForA + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B.values(Bstart + Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounterForA
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ } else {
+ while (colCounterForB < nB) {
+ var rowCounter = 0
+ val Cstart = colCounterForB * mA
+ while (rowCounter < mA) {
+ var i = Arows(rowCounter)
+ val indEnd = Arows(rowCounter + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B(colCounterForB, Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounter
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounter += 1
+ }
+ colCounterForB += 1
+ }
+ }
+ } else {
+ // Scale matrix first if `beta` is not equal to 0.0
+ if (beta != 0.0){
+ nativeBLAS.dscal(C.values.length, beta, C.values, 1)
+ }
+ // Perform matrix multiplication and add to C. The rows of A are multiplied by the columns of
+ // B, and added to C.
+ var colCounterForB = 0 // the column to be updated in C
+ if (!transB) { // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var colCounterForA = 0 // The column of A to multiply with the row of B
+ while (colCounterForA < kA){
+ var i = Acols(colCounterForA)
+ val indEnd = Acols(colCounterForA + 1)
+ val Bval = B(colCounterForA, colCounterForB)
+ val Cstart = colCounterForB * mA
--- End diff --
move this to the outer loop
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-54701366
[QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19889/consoleFull) for PR 2294 at commit [`56d7c85`](https://github.com/apache/spark/commit/56d7c85a1d58ef68c31208e062a4458e170111d3).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17709094
--- Diff: mllib/src/test/scala/org/apache/spark/mllib/linalg/BreezeMatrixConversionSuite.scala ---
@@ -37,4 +37,26 @@ class BreezeMatrixConversionSuite extends FunSuite {
assert(mat.numCols === breeze.cols)
assert(mat.values.eq(breeze.data), "should not copy data")
}
+
+ test("sparse matrix to breeze") {
+ val values = Array(1.0, 2.0, 4.0, 5.0)
+ val colPointers = Array(0, 2, 4)
--- End diff --
`colPtrs`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-54700361
[QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19875/consoleFull) for PR 2294 at commit [`41b2da3`](https://github.com/apache/spark/commit/41b2da30797e3da011e3261f8c2b89b9c1608d23).
* This patch **fails** unit tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `class SparseMatrix(val numRows: Int,`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17704457
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +201,368 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ if (alpha == 0.0) {
+ logWarning("gemm: alpha is equal to 0. Returning C.")
+ } else {
+ A match {
+ case sparse: SparseMatrix =>
+ gemm(transA, transB, alpha, sparse, B, beta, C)
+ case dense: DenseMatrix =>
+ gemm(transA, transB, alpha, dense, B, beta, C)
+ case _ =>
+ throw new IllegalArgumentException(s"gemm doesn't support matrix type ${A.getClass}.")
+ }
+ }
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ gemm(false, false, alpha, A, B, beta, C)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `DenseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: DenseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+ val tAstr = if (!transA) "N" else "T"
+ val tBstr = if (!transB) "N" else "T"
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ nativeBLAS.dgemm(tAstr, tBstr, mA, nB, kA, alpha, A.values, A.numRows, B.values, B.numRows,
+ beta, C.values, C.numRows)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `SparseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: SparseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ val Avals = A.values
+ val Arows = if (!transA) A.rowIndices else A.colPtrs
+ val Acols = if (!transA) A.colPtrs else A.rowIndices
+
+ // Slicing is easy in this case. This is the optimal multiplication setting for sparse matrices
+ if (transA){
+ var colCounterForB = 0
+ if (!transB){ // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var rowCounterForA = 0
+ val Cstart = colCounterForB * mA
+ val Bstart = colCounterForB * kA
+ while (rowCounterForA < mA) {
+ var i = Arows(rowCounterForA)
+ val indEnd = Arows(rowCounterForA + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B.values(Bstart + Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounterForA
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ } else {
+ while (colCounterForB < nB) {
+ var rowCounter = 0
+ val Cstart = colCounterForB * mA
+ while (rowCounter < mA) {
+ var i = Arows(rowCounter)
+ val indEnd = Arows(rowCounter + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B(colCounterForB, Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounter
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounter += 1
+ }
+ colCounterForB += 1
+ }
+ }
+ } else {
+ // Scale matrix first if `beta` is not equal to 0.0
+ if (beta != 0.0){
+ nativeBLAS.dscal(C.values.length, beta, C.values, 1)
+ }
+ // Perform matrix multiplication and add to C. The rows of A are multiplied by the columns of
+ // B, and added to C.
+ var colCounterForB = 0 // the column to be updated in C
+ if (!transB) { // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var colCounterForA = 0 // The column of A to multiply with the row of B
+ while (colCounterForA < kA){
+ var i = Acols(colCounterForA)
+ val indEnd = Acols(colCounterForA + 1)
+ val Bval = B(colCounterForA, colCounterForB)
+ val Cstart = colCounterForB * mA
+ while (i < indEnd){
+ C.values(Cstart + Arows(i)) += Avals(i) * Bval * alpha
+ i += 1
+ }
+ colCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ } else {
+ while (colCounterForB < nB) {
+ var colCounterForA = 0 // The column of A to multiply with the row of B
+ while (colCounterForA < kA){
+ var i = Acols(colCounterForA)
+ val indEnd = Acols(colCounterForA + 1)
+ val Bval = B(colCounterForB, colCounterForA)
+ val Cstart = colCounterForB * mA
+ while (i < indEnd){
+ C.values(Cstart + Arows(i)) += Avals(i) * Bval * alpha
+ i += 1
+ }
+ colCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ }
+ }
+ }
+
+ /**
+ * y := alpha * A * x + beta * y
+ * @param trans specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
--- End diff --
ditto: it is a boolean now
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by brkyvz <gi...@git.apache.org>.
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17709652
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +201,368 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ if (alpha == 0.0) {
+ logWarning("gemm: alpha is equal to 0. Returning C.")
+ } else {
+ A match {
+ case sparse: SparseMatrix =>
+ gemm(transA, transB, alpha, sparse, B, beta, C)
+ case dense: DenseMatrix =>
+ gemm(transA, transB, alpha, dense, B, beta, C)
+ case _ =>
+ throw new IllegalArgumentException(s"gemm doesn't support matrix type ${A.getClass}.")
+ }
+ }
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ gemm(false, false, alpha, A, B, beta, C)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `DenseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: DenseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+ val tAstr = if (!transA) "N" else "T"
+ val tBstr = if (!transB) "N" else "T"
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ nativeBLAS.dgemm(tAstr, tBstr, mA, nB, kA, alpha, A.values, A.numRows, B.values, B.numRows,
+ beta, C.values, C.numRows)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `SparseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: SparseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ val Avals = A.values
+ val Arows = if (!transA) A.rowIndices else A.colPtrs
+ val Acols = if (!transA) A.colPtrs else A.rowIndices
+
+ // Slicing is easy in this case. This is the optimal multiplication setting for sparse matrices
+ if (transA){
+ var colCounterForB = 0
+ if (!transB){ // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var rowCounterForA = 0
+ val Cstart = colCounterForB * mA
+ val Bstart = colCounterForB * kA
+ while (rowCounterForA < mA) {
+ var i = Arows(rowCounterForA)
+ val indEnd = Arows(rowCounterForA + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B.values(Bstart + Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounterForA
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ } else {
+ while (colCounterForB < nB) {
+ var rowCounter = 0
+ val Cstart = colCounterForB * mA
+ while (rowCounter < mA) {
+ var i = Arows(rowCounter)
+ val indEnd = Arows(rowCounter + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B(colCounterForB, Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounter
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounter += 1
+ }
+ colCounterForB += 1
+ }
+ }
+ } else {
+ // Scale matrix first if `beta` is not equal to 0.0
+ if (beta != 0.0){
+ nativeBLAS.dscal(C.values.length, beta, C.values, 1)
+ }
+ // Perform matrix multiplication and add to C. The rows of A are multiplied by the columns of
+ // B, and added to C.
+ var colCounterForB = 0 // the column to be updated in C
+ if (!transB) { // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var colCounterForA = 0 // The column of A to multiply with the row of B
+ while (colCounterForA < kA){
+ var i = Acols(colCounterForA)
+ val indEnd = Acols(colCounterForA + 1)
+ val Bval = B(colCounterForA, colCounterForB)
+ val Cstart = colCounterForB * mA
+ while (i < indEnd){
+ C.values(Cstart + Arows(i)) += Avals(i) * Bval * alpha
+ i += 1
+ }
+ colCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ } else {
+ while (colCounterForB < nB) {
+ var colCounterForA = 0 // The column of A to multiply with the row of B
+ while (colCounterForA < kA){
+ var i = Acols(colCounterForA)
+ val indEnd = Acols(colCounterForA + 1)
+ val Bval = B(colCounterForB, colCounterForA)
+ val Cstart = colCounterForB * mA
+ while (i < indEnd){
+ C.values(Cstart + Arows(i)) += Avals(i) * Bval * alpha
+ i += 1
+ }
+ colCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ }
+ }
+ }
+
+ /**
+ * y := alpha * A * x + beta * y
+ * @param trans specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ * @param beta a scalar that can be used to scale vector y.
+ * @param y the resulting vector y. Size of m x 1.
+ */
+ def gemv(
+ trans: Boolean,
+ alpha: Double,
+ A: Matrix,
+ x: DenseVector,
+ beta: Double,
+ y: DenseVector): Unit = {
+
+ val mA: Int = if (!trans) A.numRows else A.numCols
+ val nx: Int = x.size
+ val nA: Int = if (!trans) A.numCols else A.numRows
+
+ require(nA == nx, s"The columns of A don't match the number of elements of x. A: $nA, x: $nx")
+ require(mA == y.size,
+ s"The rows of A don't match the number of elements of y. A: $mA, y:${y.size}}")
+ if (alpha == 0.0) {
+ logWarning("gemv: alpha is equal to 0. Returning y.")
+ } else {
+ A match {
+ case sparse: SparseMatrix =>
+ gemv(trans, alpha, sparse, x, beta, y)
+ case dense: DenseMatrix =>
+ gemv(trans, alpha, dense, x, beta, y)
+ case _ =>
+ throw new IllegalArgumentException(s"gemv doesn't support matrix type ${A.getClass}.")
+ }
+ }
+ }
+
+ /**
+ * y := alpha * A * x + beta * y
+ *
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ * @param beta a scalar that can be used to scale vector y.
+ * @param y the resulting vector y. Size of m x 1.
+ */
+ def gemv(
+ alpha: Double,
+ A: Matrix,
+ x: DenseVector,
+ beta: Double,
+ y: DenseVector): Unit = {
+ gemv(false, alpha, A, x, beta, y)
+ }
+
+ /**
+ * y := alpha * A * x
+ *
+ * @param trans specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ *
+ * @return `DenseVector` y, the result of the matrix-vector multiplication. Size of m x 1.
+ */
+ def gemv(
+ trans: Boolean,
+ alpha: Double,
+ A: Matrix,
+ x: DenseVector): DenseVector = {
+ val m = if(!trans) A.numRows else A.numCols
+
+ val y: DenseVector = new DenseVector(Array.fill(m)(0.0))
+ gemv(trans, alpha, A, x, 0.0, y)
+
+ y
+ }
+
+ /**
+ * y := alpha * A * x
+ *
+ * @param alpha a scalar to scale the multiplication A * x.
+ * @param A the matrix A that will be left multiplied to x. Size of m x n.
+ * @param x the vector x that will be left multiplied by A. Size of n x 1.
+ *
+ * @return `DenseVector` y, the result of the matrix-vector multiplication. Size of m x 1.
+ */
+ def gemv(
+ alpha: Double,
+ A: Matrix,
+ x: DenseVector): DenseVector = {
+ gemv(false, alpha, A, x)
+ }
+
+
+ /**
+ * y := alpha * A * x + beta * y
+ * For `DenseMatrix` A.
+ */
+ private def gemv(
+ trans: Boolean,
+ alpha: Double,
+ A: DenseMatrix,
+ x: DenseVector,
+ beta: Double,
+ y: DenseVector): Unit = {
+ val tStrA = if (!trans) "N" else "T"
+ nativeBLAS.dgemv(tStrA, A.numRows, A.numCols, alpha, A.values, A.numRows, x.values, 1, beta,
--- End diff --
I'll take a look at it
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17709082
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala ---
@@ -59,11 +93,113 @@ trait Matrix extends Serializable {
*/
class DenseMatrix(val numRows: Int, val numCols: Int, val values: Array[Double]) extends Matrix {
- require(values.length == numRows * numCols)
+ require(values.length == numRows * numCols, "The number of values supplied doesn't match the " +
+ s"size of the matrix! values.length: ${values.length}, numRows * numCols: ${numRows * numCols}")
override def toArray: Array[Double] = values
- private[mllib] override def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+ private[mllib] def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+
+ private[mllib] def apply(i: Int): Double = values(i)
+
+ private[mllib] def apply(i: Int, j: Int): Double = values(index(i, j))
+
+ private[mllib] def index(i: Int, j: Int): Int = i + numRows * j
+
+ private[mllib] def update(i: Int, j: Int, v: Double){
+ values(index(i, j)) = v
+ }
+
+ override def copy = new DenseMatrix(numRows, numCols, values.clone())
+}
+
+/**
+ * Column-majored sparse matrix.
+ * The entry values are stored in Compressed Sparse Column (CSC) format.
+ * For example, the following matrix
+ * {{{
+ * 1.0 0.0 4.0
+ * 0.0 3.0 5.0
+ * 2.0 0.0 6.0
+ * }}}
+ * is stored as `values: [1.0, 2.0, 3.0, 4.0, 5.0, 6.0]`,
+ * `rowIndices=[0, 2, 1, 0, 1, 2]`, `colPointers=[0, 2, 3, 6]`.
+ *
+ * @param numRows number of rows
+ * @param numCols number of columns
+ * @param colPtrs the index corresponding to the start of a new column
+ * @param rowIndices the row index of the entry
+ * @param values non-zero matrix entries in column major
+ */
+class SparseMatrix(
+ val numRows: Int,
+ val numCols: Int,
+ val colPtrs: Array[Int],
+ val rowIndices: Array[Int],
+ val values: Array[Double]) extends Matrix {
+
+ require(values.length == rowIndices.length, "The number of row indices and values don't match! " +
+ s"values.length: ${values.length}, rowIndices.length: ${rowIndices.length}")
+ require(colPtrs.length == numCols + 1, "The length of the column indices should be the " +
+ s"number of columns + 1. Currently, colPointers.length: ${colPtrs.length}, " +
+ s"numCols: $numCols")
+
+ override def toArray: Array[Double] = {
+ val arr = Array.fill(numRows * numCols)(0.0)
--- End diff --
`new Array[Double](numRows * numCols)`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2294#issuecomment-56006229
[QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20527/consoleFull) for PR 2294 at commit [`f79db9c`](https://github.com/apache/spark/commit/f79db9c0d82ceea41d59972e3bde9fa2a17b6112).
* This patch **fails** unit tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `sealed trait Matrix extends Serializable `
* `class SparseMatrix(`
* `sealed trait Vector extends Serializable `
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17704447
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala ---
@@ -197,4 +201,368 @@ private[mllib] object BLAS extends Serializable {
throw new IllegalArgumentException(s"scal doesn't support vector type ${x.getClass}.")
}
}
+
+ // For level-3 routines, we use the native BLAS.
+ private def nativeBLAS: NetlibBLAS = {
+ if (_nativeBLAS == null) {
+ _nativeBLAS = NativeBLAS
+ }
+ _nativeBLAS
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * @param transA specify whether to use matrix A, or the transpose of matrix A. Should be "N" or
+ * "n" to use A, and "T" or "t" to use the transpose of A.
+ * @param transB specify whether to use matrix B, or the transpose of matrix B. Should be "N" or
+ * "n" to use B, and "T" or "t" to use the transpose of B.
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ if (alpha == 0.0) {
+ logWarning("gemm: alpha is equal to 0. Returning C.")
+ } else {
+ A match {
+ case sparse: SparseMatrix =>
+ gemm(transA, transB, alpha, sparse, B, beta, C)
+ case dense: DenseMatrix =>
+ gemm(transA, transB, alpha, dense, B, beta, C)
+ case _ =>
+ throw new IllegalArgumentException(s"gemm doesn't support matrix type ${A.getClass}.")
+ }
+ }
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ *
+ * @param alpha a scalar to scale the multiplication A * B.
+ * @param A the matrix A that will be left multiplied to B. Size of m x k.
+ * @param B the matrix B that will be left multiplied by A. Size of k x n.
+ * @param beta a scalar that can be used to scale matrix C.
+ * @param C the resulting matrix C. Size of m x n.
+ */
+ def gemm(
+ alpha: Double,
+ A: Matrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ gemm(false, false, alpha, A, B, beta, C)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `DenseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: DenseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+ val tAstr = if (!transA) "N" else "T"
+ val tBstr = if (!transB) "N" else "T"
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ nativeBLAS.dgemm(tAstr, tBstr, mA, nB, kA, alpha, A.values, A.numRows, B.values, B.numRows,
+ beta, C.values, C.numRows)
+ }
+
+ /**
+ * C := alpha * A * B + beta * C
+ * For `SparseMatrix` A.
+ */
+ private def gemm(
+ transA: Boolean,
+ transB: Boolean,
+ alpha: Double,
+ A: SparseMatrix,
+ B: DenseMatrix,
+ beta: Double,
+ C: DenseMatrix): Unit = {
+ val mA: Int = if (!transA) A.numRows else A.numCols
+ val nB: Int = if (!transB) B.numCols else B.numRows
+ val kA: Int = if (!transA) A.numCols else A.numRows
+ val kB: Int = if (!transB) B.numRows else B.numCols
+
+ require(kA == kB, s"The columns of A don't match the rows of B. A: $kA, B: $kB")
+ require(mA == C.numRows, s"The rows of C don't match the rows of A. C: ${C.numRows}, A: $mA")
+ require(nB == C.numCols,
+ s"The columns of C don't match the columns of B. C: ${C.numCols}, A: $nB")
+
+ val Avals = A.values
+ val Arows = if (!transA) A.rowIndices else A.colPtrs
+ val Acols = if (!transA) A.colPtrs else A.rowIndices
+
+ // Slicing is easy in this case. This is the optimal multiplication setting for sparse matrices
+ if (transA){
+ var colCounterForB = 0
+ if (!transB){ // Expensive to put the check inside the loop
+ while (colCounterForB < nB) {
+ var rowCounterForA = 0
+ val Cstart = colCounterForB * mA
+ val Bstart = colCounterForB * kA
+ while (rowCounterForA < mA) {
+ var i = Arows(rowCounterForA)
+ val indEnd = Arows(rowCounterForA + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B.values(Bstart + Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounterForA
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounterForA += 1
+ }
+ colCounterForB += 1
+ }
+ } else {
+ while (colCounterForB < nB) {
+ var rowCounter = 0
+ val Cstart = colCounterForB * mA
+ while (rowCounter < mA) {
+ var i = Arows(rowCounter)
+ val indEnd = Arows(rowCounter + 1)
+ var sum = 0.0
+ while (i < indEnd) {
+ sum += Avals(i) * B(colCounterForB, Acols(i))
+ i += 1
+ }
+ val Cindex = Cstart + rowCounter
+ C.values(Cindex) = beta * C.values(Cindex) + sum * alpha
+ rowCounter += 1
+ }
+ colCounterForB += 1
+ }
+ }
+ } else {
+ // Scale matrix first if `beta` is not equal to 0.0
+ if (beta != 0.0){
+ nativeBLAS.dscal(C.values.length, beta, C.values, 1)
--- End diff --
Use `f2jBLAS` instead. This is a level-1 operation.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3418] Sparse Matrix support (CCS) and a...
Posted by mengxr <gi...@git.apache.org>.
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/2294#discussion_r17255329
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala ---
@@ -63,7 +88,162 @@ class DenseMatrix(val numRows: Int, val numCols: Int, val values: Array[Double])
override def toArray: Array[Double] = values
- private[mllib] override def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+ private [mllib] def toBreeze: BM[Double] = new BDM[Double](numRows, numCols, values)
+
+ private [mllib] def apply(i: Int): Double = values(i)
+
+ private [mllib] def apply(r: Int, c: Int): Double = values(index(r, c))
+
+ private [mllib] def index(r: Int, c: Int): Int = r + numRows * c
+
+ private [mllib] def update(r: Int, c: Int, v: Double){
+ values(index(r, c)) = v
+ }
+
+ def copy = new DenseMatrix(numRows, numCols, values.clone())
+}
+
+/**
+ * Factory methods for [[org.apache.spark.mllib.linalg.DenseMatrix]].
+ *
+ * These methods can be used to generate common matrix types such as the Identity matrix, any
+ * diagonal matrix, zero matrix, and random matrices.
+ */
+object DenseMatrix {
+
+ /**
+ * Generate a `DenseMatrix` consisting of zeros.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values of zeros
+ */
+ def zeros(numRows: Int, numCols: Int) =
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(0.0))
+
+ /**
+ * Generate a `DenseMatrix` consisting of ones.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values of ones
+ */
+ def ones(numRows: Int, numCols: Int) =
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(1.0))
+
+ /**
+ * Generate an Identity Matrix in `DenseMatrix` format.
+ * @param n number of rows and columns of the matrix
+ * @return `DenseMatrix` with size `n` x `n` and values of ones on the diagonal
+ */
+ def eye(n: Int) = {
+ val identity = DenseMatrix.zeros(n,n)
+ for (i <- 0 until n){
+ identity.update(i, i, 1.0)
+ }
+ identity
+ }
+
+ /**
+ * Generate a `DenseMatrix` consisting of i.i.d. uniform random numbers.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values in U(0, 1)
+ */
+ def rand(numRows: Int, numCols: Int) = {
+ val rand = new scala.util.Random
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(rand.nextDouble()))
+ }
+
+ /**
+ * Generate a `DenseMatrix` consisting of i.i.d. gaussian random numbers.
+ * @param numRows number of rows of the matrix
+ * @param numCols number of columns of the matrix
+ * @return `DenseMatrix` with size `numRows` x `numCols` and values in N(0, 1)
+ */
+ def randn(numRows: Int, numCols: Int) = {
+ val rand = new scala.util.Random
+ new DenseMatrix(numRows, numCols, Array.fill(numRows * numCols)(rand.nextGaussian()))
+ }
+
+ /**
+ * Generate a diagonal matrix in `DenseMatrix` format from the supplied values.
+ * @param values values on the diagonal of the matrix
+ * @return Square `DenseMatrix` with size `values.length` x `values.length` and `values`
+ * on the diagonal
+ */
+ def diag(values: Array[Double]) = {
+ val n = values.length
+ val matrix = DenseMatrix.eye(n)
+ for (i <- 0 until n) matrix.update(i, i, values(i))
--- End diff --
`for` -> `while`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org