You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by hv...@apache.org on 2016/04/19 23:30:48 UTC
spark git commit: [SPARK-14656][CORE] Fix
Benchmark.getPorcessorName() always return "Unknown processor" on Linux
Repository: spark
Updated Branches:
refs/heads/master 5cb2e3360 -> 0b8369d85
[SPARK-14656][CORE] Fix Benchmark.getPorcessorName() always return "Unknown processor" on Linux
## What changes were proposed in this pull request?
This PR returns correct processor name in ```/proc/cpuinfo``` on Linux from ```Benchmark.getPorcessorName()```. Now, this return ```Unknown processor```.
Since ```Utils.executeAndGetOutput(Seq("which", "grep"))``` return ```/bin/grep\n```, it is failed to execute ```/bin/grep\n```. This PR strips ```\n``` at the end of the line of a result of ```Utils.executeAndGetOutput()```
Before applying this PR
````
Java HotSpot(TM) 64-Bit Server VM 1.8.0_66-b17 on Linux 2.6.32-504.el6.x86_64
Unknown processor
back-to-back filter: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------
Dataset 472 / 503 21.2 47.2 1.0X
DataFrame 51 / 58 198.0 5.1 9.3X
RDD 189 / 211 52.8 18.9 2.5X
````
After applying this PR
```
Java HotSpot(TM) 64-Bit Server VM 1.8.0_66-b17 on Linux 2.6.32-504.el6.x86_64
Intel(R) Xeon(R) CPU E5-2667 v2 3.30GHz
back-to-back filter: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------
Dataset 490 / 502 20.4 49.0 1.0X
DataFrame 55 / 61 183.4 5.5 9.0X
RDD 210 / 237 47.7 21.0 2.3X
```
## How was this patch tested?
Run Benchmark programs on Linux by hand
Author: Kazuaki Ishizaki <is...@jp.ibm.com>
Closes #12411 from kiszk/SPARK-14656.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/0b8369d8
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/0b8369d8
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/0b8369d8
Branch: refs/heads/master
Commit: 0b8369d8548c0204b9c24d826c731063b72360b8
Parents: 5cb2e33
Author: Kazuaki Ishizaki <is...@jp.ibm.com>
Authored: Tue Apr 19 23:30:34 2016 +0200
Committer: Herman van Hovell <hv...@questtec.nl>
Committed: Tue Apr 19 23:30:34 2016 +0200
----------------------------------------------------------------------
core/src/main/scala/org/apache/spark/util/Benchmark.scala | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/0b8369d8/core/src/main/scala/org/apache/spark/util/Benchmark.scala
----------------------------------------------------------------------
diff --git a/core/src/main/scala/org/apache/spark/util/Benchmark.scala b/core/src/main/scala/org/apache/spark/util/Benchmark.scala
index 9e40baf..3718542 100644
--- a/core/src/main/scala/org/apache/spark/util/Benchmark.scala
+++ b/core/src/main/scala/org/apache/spark/util/Benchmark.scala
@@ -96,9 +96,9 @@ private[spark] object Benchmark {
Utils.executeAndGetOutput(Seq("/usr/sbin/sysctl", "-n", "machdep.cpu.brand_string"))
} else if (SystemUtils.IS_OS_LINUX) {
Try {
- val grepPath = Utils.executeAndGetOutput(Seq("which", "grep"))
+ val grepPath = Utils.executeAndGetOutput(Seq("which", "grep")).stripLineEnd
Utils.executeAndGetOutput(Seq(grepPath, "-m", "1", "model name", "/proc/cpuinfo"))
- .replaceFirst("model name[\\s*]:[\\s*]", "")
+ .stripLineEnd.replaceFirst("model name[\\s*]:[\\s*]", "")
}.getOrElse("Unknown processor")
} else {
System.getenv("PROCESSOR_IDENTIFIER")
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org