You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by me...@apache.org on 2014/10/30 08:06:09 UTC
git commit: [SPARK-4130][MLlib] Fixing libSVM parser bug with extra
whitespace
Repository: spark
Updated Branches:
refs/heads/master 6db315746 -> c7ad08520
[SPARK-4130][MLlib] Fixing libSVM parser bug with extra whitespace
This simple patch filters out extra whitespace entries.
Author: Joseph E. Gonzalez <jo...@gmail.com>
Author: Joey <jo...@gmail.com>
Closes #2996 from jegonzal/loadLibSVM and squashes the following commits:
e0227ab [Joey] improving readability
e028e84 [Joseph E. Gonzalez] fixing whitespace bug in loadLibSVMFile when parsing libSVM files
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/c7ad0852
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/c7ad0852
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/c7ad0852
Branch: refs/heads/master
Commit: c7ad0852084dc28f3ebc144adfd4928b23f1c8ea
Parents: 6db3157
Author: Joseph E. Gonzalez <jo...@gmail.com>
Authored: Thu Oct 30 00:05:57 2014 -0700
Committer: Xiangrui Meng <me...@databricks.com>
Committed: Thu Oct 30 00:05:57 2014 -0700
----------------------------------------------------------------------
mllib/src/main/scala/org/apache/spark/mllib/util/MLUtils.scala | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/c7ad0852/mllib/src/main/scala/org/apache/spark/mllib/util/MLUtils.scala
----------------------------------------------------------------------
diff --git a/mllib/src/main/scala/org/apache/spark/mllib/util/MLUtils.scala b/mllib/src/main/scala/org/apache/spark/mllib/util/MLUtils.scala
index dce0adf..b88e08b 100644
--- a/mllib/src/main/scala/org/apache/spark/mllib/util/MLUtils.scala
+++ b/mllib/src/main/scala/org/apache/spark/mllib/util/MLUtils.scala
@@ -76,7 +76,7 @@ object MLUtils {
.map { line =>
val items = line.split(' ')
val label = items.head.toDouble
- val (indices, values) = items.tail.map { item =>
+ val (indices, values) = items.tail.filter(_.nonEmpty).map { item =>
val indexAndValue = item.split(':')
val index = indexAndValue(0).toInt - 1 // Convert 1-based indices to 0-based.
val value = indexAndValue(1).toDouble
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org