You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by sr...@apache.org on 2015/02/08 11:13:34 UTC
spark git commit: [SPARK-5656] Fail gracefully for large values of k
and/or n that will ex...
Repository: spark
Updated Branches:
refs/heads/master 6fb141e2a -> 487831369
[SPARK-5656] Fail gracefully for large values of k and/or n that will ex...
...ceed max int.
Large values of k and/or n in EigenValueDecomposition.symmetricEigs will result in array initialization to a value larger than Integer.MAX_VALUE in the following: var v = new Array[Double](n * ncv)
Author: mbittmann <mb...@gmail.com>
Author: bittmannm <ma...@agilex.com>
Closes #4433 from mbittmann/master and squashes the following commits:
ee56e05 [mbittmann] [SPARK-5656] Combine checks into simple message
e49cbbb [mbittmann] [SPARK-5656] Simply error message
860836b [mbittmann] Array size check updates based on code review
a604816 [bittmannm] [SPARK-5656] Fail gracefully for large values of k and/or n that will exceed max int.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/48783136
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/48783136
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/48783136
Branch: refs/heads/master
Commit: 48783136958e76d96f477802805e000ee5da5697
Parents: 6fb141e
Author: mbittmann <mb...@gmail.com>
Authored: Sun Feb 8 10:13:29 2015 +0000
Committer: Sean Owen <so...@cloudera.com>
Committed: Sun Feb 8 10:13:29 2015 +0000
----------------------------------------------------------------------
.../org/apache/spark/mllib/linalg/EigenValueDecomposition.scala | 3 +++
1 file changed, 3 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/48783136/mllib/src/main/scala/org/apache/spark/mllib/linalg/EigenValueDecomposition.scala
----------------------------------------------------------------------
diff --git a/mllib/src/main/scala/org/apache/spark/mllib/linalg/EigenValueDecomposition.scala b/mllib/src/main/scala/org/apache/spark/mllib/linalg/EigenValueDecomposition.scala
index 3515461..9d6f975 100644
--- a/mllib/src/main/scala/org/apache/spark/mllib/linalg/EigenValueDecomposition.scala
+++ b/mllib/src/main/scala/org/apache/spark/mllib/linalg/EigenValueDecomposition.scala
@@ -79,6 +79,9 @@ private[mllib] object EigenValueDecomposition {
// Mode 1: A*x = lambda*x, A symmetric
iparam(6) = 1
+ require(n * ncv.toLong <= Integer.MAX_VALUE && ncv * (ncv.toLong + 8) <= Integer.MAX_VALUE,
+ s"k = $k and/or n = $n are too large to compute an eigendecomposition")
+
var ido = new intW(0)
var info = new intW(0)
var resid = new Array[Double](n)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org