You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by chetkhatri <gi...@git.apache.org> on 2017/12/24 08:35:42 UTC
[GitHub] spark pull request #20070: SPARK-22896 Improvement in String interpolation
GitHub user chetkhatri opened a pull request:
https://github.com/apache/spark/pull/20070
SPARK-22896 Improvement in String interpolation
## What changes were proposed in this pull request?
* String interpolation in ml pipeline example has been corrected as per scala standard.
## How was this patch tested?
* manually tested.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/chetkhatri/spark mllib-chetan-contrib
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20070.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20070
----
commit 9916fd1f67234b1fa5608231181bdf3b08718981
Author: chetkhatri <ck...@...>
Date: 2017-12-24T08:33:49Z
SPARK-22896 Improvement in String interpolation
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20070: SPARK-22896 Improvement in String interpolation
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20070
**[Test build #4033 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4033/testReport)** for PR 20070 at commit [`e891f53`](https://github.com/apache/spark/commit/e891f53dfd609ef9cb46f70147c59f1113c7c0cb).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20070: SPARK-22896 Improvement in String interpolation
Posted by chetkhatri <gi...@git.apache.org>.
Github user chetkhatri commented on a diff in the pull request:
https://github.com/apache/spark/pull/20070#discussion_r158975980
--- Diff: examples/src/main/scala/org/apache/spark/examples/ml/ChiSquareTestExample.scala ---
@@ -52,9 +52,9 @@ object ChiSquareTestExample {
val df = data.toDF("label", "features")
val chi = ChiSquareTest.test(df, "features", "label").head
- println("pValues = " + chi.getAs[Vector](0))
- println("degreesOfFreedom = " + chi.getSeq[Int](1).mkString("[", ",", "]"))
- println("statistics = " + chi.getAs[Vector](2))
+ println(s"pValues = ${chi.getAs[Vector](0)}")
--- End diff --
Ok
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20070: SPARK-22896 Improvement in String interpolation
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/20070#discussion_r159143849
--- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/LatentDirichletAllocationExample.scala ---
@@ -46,7 +46,10 @@ object LatentDirichletAllocationExample {
val topics = ldaModel.topicsMatrix
for (topic <- Range(0, 3)) {
print(s"Topic $topic :")
- for (word <- Range(0, ldaModel.vocabSize)) { print(s" ${topics(word, topic)}") }
+ for (word <- Range(0, ldaModel.vocabSize))
+ {
--- End diff --
No, we put the open brace on the preceding line with for, and don't triple indent. See any other for loop in the code.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20070: SPARK-22896 Improvement in String interpolation
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20070
**[Test build #4033 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4033/testReport)** for PR 20070 at commit [`e891f53`](https://github.com/apache/spark/commit/e891f53dfd609ef9cb46f70147c59f1113c7c0cb).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20070: SPARK-22896 Improvement in String interpolation
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20070
**[Test build #4026 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4026/testReport)** for PR 20070 at commit [`5507cad`](https://github.com/apache/spark/commit/5507cad0516b5bb4a6fb157a32510b30e85176f6).
* This patch **fails Scala style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20070: SPARK-22896 Improvement in String interpolation
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/20070#discussion_r159125085
--- Diff: examples/src/main/scala/org/apache/spark/examples/ml/VectorIndexerExample.scala ---
@@ -41,8 +41,8 @@ object VectorIndexerExample {
val indexerModel = indexer.fit(data)
val categoricalFeatures: Set[Int] = indexerModel.categoryMaps.keys.toSet
- println(s"Chose ${categoricalFeatures.size} categorical features: " +
- categoricalFeatures.mkString(", "))
+ println(s"Chose ${categoricalFeatures.size} " +
+ s"categorical features: {$categoricalFeatures.mkString(", ")}")
--- End diff --
`./dev/run-tests`
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20070: SPARK-22896 Improvement in String interpolation
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/20070#discussion_r159129816
--- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/LatentDirichletAllocationExample.scala ---
@@ -42,11 +42,11 @@ object LatentDirichletAllocationExample {
val ldaModel = new LDA().setK(3).run(corpus)
// Output topics. Each is a distribution over words (matching word count vectors)
- println("Learned topics (as distributions over vocab of " + ldaModel.vocabSize + " words):")
+ println(s"Learned topics (as distributions over vocab of ${ldaModel.vocabSize} words):")
val topics = ldaModel.topicsMatrix
for (topic <- Range(0, 3)) {
- print("Topic " + topic + ":")
- for (word <- Range(0, ldaModel.vocabSize)) { print(" " + topics(word, topic)); }
+ print(s"Topic $topic :")
+ for (word <- Range(0, ldaModel.vocabSize)) { print(s" ${topics(word, topic)}") }
--- End diff --
Go ahead and put the print on a new line (I know it wasn't before)
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20070: SPARK-22896 Improvement in String interpolation
Posted by chetkhatri <gi...@git.apache.org>.
Github user chetkhatri commented on a diff in the pull request:
https://github.com/apache/spark/pull/20070#discussion_r158976171
--- Diff: examples/src/main/scala/org/apache/spark/examples/ml/DeveloperApiExample.scala ---
@@ -169,10 +169,10 @@ private class MyLogisticRegressionModel(
Vectors.dense(-margin, margin)
}
- /** Number of classes the label can take. 2 indicates binary classification. */
+ // Number of classes the label can take. 2 indicates binary classification.
--- End diff --
+1
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20070: SPARK-22896 Improvement in String interpolation
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/20070#discussion_r158744360
--- Diff: examples/src/main/scala/org/apache/spark/examples/streaming/CustomReceiver.scala ---
@@ -82,9 +82,9 @@ class CustomReceiver(host: String, port: Int)
var socket: Socket = null
var userInput: String = null
try {
- logInfo("Connecting to " + host + ":" + port)
+ logInfo(s"Connecting to $host $port")
--- End diff --
Nit: we could make the string consistent with the one two lines below by adding a colon
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20070: SPARK-22896 Improvement in String interpolation
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20070
**[Test build #4028 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4028/testReport)** for PR 20070 at commit [`79e6789`](https://github.com/apache/spark/commit/79e6789e1c7621b374785e036f7132c37da6da21).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20070: SPARK-22896 Improvement in String interpolation
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/20070#discussion_r158744210
--- Diff: examples/src/main/scala/org/apache/spark/examples/ml/ChiSquareTestExample.scala ---
@@ -52,9 +52,9 @@ object ChiSquareTestExample {
val df = data.toDF("label", "features")
val chi = ChiSquareTest.test(df, "features", "label").head
- println("pValues = " + chi.getAs[Vector](0))
- println("degreesOfFreedom = " + chi.getSeq[Int](1).mkString("[", ",", "]"))
- println("statistics = " + chi.getAs[Vector](2))
+ println(s"pValues = ${chi.getAs[Vector](0)}")
--- End diff --
I think this is OK; anything more complex I might suggest breaking out the expression into a val.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20070: SPARK-22896 Improvement in String interpolation
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/20070
OK, we can do this for consistency. In general we don't spend a ton of time on changes like this, because there is already way too much else to review. Please make one change only (this one) that updates all examples in one go.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20070: SPARK-22896 Improvement in String interpolation
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20070
**[Test build #4028 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4028/testReport)** for PR 20070 at commit [`79e6789`](https://github.com/apache/spark/commit/79e6789e1c7621b374785e036f7132c37da6da21).
* This patch **fails Scala style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20070: SPARK-22896 Improvement in String interpolation
Posted by chetkhatri <gi...@git.apache.org>.
Github user chetkhatri commented on the issue:
https://github.com/apache/spark/pull/20070
@srowen also i did merge another similiar PR with graphx to here. so Just FYI - we are good.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20070: SPARK-22896 Improvement in String interpolation
Posted by chetkhatri <gi...@git.apache.org>.
Github user chetkhatri commented on a diff in the pull request:
https://github.com/apache/spark/pull/20070#discussion_r158978102
--- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/AssociationRulesExample.scala ---
@@ -42,14 +42,13 @@ object AssociationRulesExample {
val results = ar.run(freqItemsets)
results.collect().foreach { rule =>
- println("[" + rule.antecedent.mkString(",")
- + "=>"
- + rule.consequent.mkString(",") + "]," + rule.confidence)
+ println(s"[${rule.antecedent.mkString(",")}=>${rule.consequent.mkString(",")} ]" +
+ s" ${rule.confidence}")
}
// $example off$
sc.stop()
}
}
-// scalastyle:on println
+// scalastyle:on println
--- End diff --
Done, addressed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20070: SPARK-22896 Improvement in String interpolation
Posted by chetkhatri <gi...@git.apache.org>.
Github user chetkhatri commented on a diff in the pull request:
https://github.com/apache/spark/pull/20070#discussion_r158978648
--- Diff: examples/src/main/scala/org/apache/spark/examples/streaming/CustomReceiver.scala ---
@@ -82,9 +82,9 @@ class CustomReceiver(host: String, port: Int)
var socket: Socket = null
var userInput: String = null
try {
- logInfo("Connecting to " + host + ":" + port)
+ logInfo(s"Connecting to $host $port")
--- End diff --
Done, addressed
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20070: SPARK-22896 Improvement in String interpolation
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/20070
I think there are many more examples of this; a quick search suggests there are about 40 example files that have some string concatenation.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20070: SPARK-22896 Improvement in String interpolation
Posted by chetkhatri <gi...@git.apache.org>.
Github user chetkhatri commented on a diff in the pull request:
https://github.com/apache/spark/pull/20070#discussion_r159192570
--- Diff: examples/src/main/scala/org/apache/spark/examples/ml/QuantileDiscretizerExample.scala ---
@@ -45,7 +45,7 @@ object QuantileDiscretizerExample {
.setNumBuckets(3)
val result = discretizer.fit(df).transform(df)
- result.show()
+ result.show(false)
--- End diff --
@srowen correct either way it works for ex. examples/ml/LDAExamples.scala
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20070: SPARK-22896 Improvement in String interpolation
Posted by chetkhatri <gi...@git.apache.org>.
Github user chetkhatri commented on a diff in the pull request:
https://github.com/apache/spark/pull/20070#discussion_r159134414
--- Diff: examples/src/main/scala/org/apache/spark/examples/ml/QuantileDiscretizerExample.scala ---
@@ -45,7 +45,7 @@ object QuantileDiscretizerExample {
.setNumBuckets(3)
val result = discretizer.fit(df).transform(df)
- result.show()
+ result.show(false)
--- End diff --
We're following same style in other examples so it is good to do.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20070: SPARK-22896 Improvement in String interpolation
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20070
**[Test build #4026 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4026/testReport)** for PR 20070 at commit [`5507cad`](https://github.com/apache/spark/commit/5507cad0516b5bb4a6fb157a32510b30e85176f6).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20070: SPARK-22896 Improvement in String interpolation
Posted by chetkhatri <gi...@git.apache.org>.
Github user chetkhatri commented on a diff in the pull request:
https://github.com/apache/spark/pull/20070#discussion_r158974779
--- Diff: examples/src/main/scala/org/apache/spark/examples/graphx/Analytics.scala ---
@@ -145,9 +145,9 @@ object Analytics extends Logging {
// TriangleCount requires the graph to be partitioned
.partitionBy(partitionStrategy.getOrElse(RandomVertexCut)).cache()
val triangles = TriangleCount.run(graph)
- println("Triangles: " + triangles.vertices.map {
+ println(s"Triangles: ${triangles.vertices.map {
--- End diff --
sure
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20070: SPARK-22896 Improvement in String interpolation
Posted by chetkhatri <gi...@git.apache.org>.
Github user chetkhatri commented on the issue:
https://github.com/apache/spark/pull/20070
@srowen Request for review when you get on this.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20070: SPARK-22896 Improvement in String interpolation
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20070
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20070: SPARK-22896 Improvement in String interpolation
Posted by chetkhatri <gi...@git.apache.org>.
Github user chetkhatri commented on a diff in the pull request:
https://github.com/apache/spark/pull/20070#discussion_r159152519
--- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/LatentDirichletAllocationExample.scala ---
@@ -46,7 +46,10 @@ object LatentDirichletAllocationExample {
val topics = ldaModel.topicsMatrix
for (topic <- Range(0, 3)) {
print(s"Topic $topic :")
- for (word <- Range(0, ldaModel.vocabSize)) { print(s" ${topics(word, topic)}") }
+ for (word <- Range(0, ldaModel.vocabSize))
+ {
--- End diff --
@srowen sure done.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20070: SPARK-22896 Improvement in String interpolation
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/20070
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20070: SPARK-22896 Improvement in String interpolation
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/20070#discussion_r158744230
--- Diff: examples/src/main/scala/org/apache/spark/examples/ml/CorrelationExample.scala ---
@@ -51,10 +51,10 @@ object CorrelationExample {
val df = data.map(Tuple1.apply).toDF("features")
val Row(coeff1: Matrix) = Correlation.corr(df, "features").head
- println("Pearson correlation matrix:\n" + coeff1.toString)
+ println(s"Pearson correlation matrix:\n ${coeff1.toString}")
--- End diff --
Another thing we could improve: `.toString` is redundant here I believe
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20070: SPARK-22896 Improvement in String interpolation
Posted by chetkhatri <gi...@git.apache.org>.
Github user chetkhatri commented on a diff in the pull request:
https://github.com/apache/spark/pull/20070#discussion_r158976219
--- Diff: examples/src/main/scala/org/apache/spark/examples/ml/QuantileDiscretizerExample.scala ---
@@ -31,12 +31,11 @@ object QuantileDiscretizerExample {
// $example on$
val data = Array((0, 18.0), (1, 19.0), (2, 8.0), (3, 5.0), (4, 2.2))
- val df = spark.createDataFrame(data).toDF("id", "hour")
+ val df = spark.createDataFrame(data).toDF("id", "hour").repartition(1)
--- End diff --
ok
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20070: SPARK-22896 Improvement in String interpolation
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/20070#discussion_r158744255
--- Diff: examples/src/main/scala/org/apache/spark/examples/ml/DeveloperApiExample.scala ---
@@ -169,10 +169,10 @@ private class MyLogisticRegressionModel(
Vectors.dense(-margin, margin)
}
- /** Number of classes the label can take. 2 indicates binary classification. */
+ // Number of classes the label can take. 2 indicates binary classification.
--- End diff --
Yeah, good to make this a standard comment, not scaladoc style
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20070: SPARK-22896 Improvement in String interpolation
Posted by chetkhatri <gi...@git.apache.org>.
Github user chetkhatri commented on a diff in the pull request:
https://github.com/apache/spark/pull/20070#discussion_r159134489
--- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/HypothesisTestingExample.scala ---
@@ -68,7 +68,7 @@ object HypothesisTestingExample {
// against the label.
val featureTestResults: Array[ChiSqTestResult] = Statistics.chiSqTest(obs)
featureTestResults.zipWithIndex.foreach { case (k, v) =>
- println("Column " + (v + 1).toString + ":")
+ println(s"Column ${(v + 1).toString} :")
--- End diff --
@srowen Thanks , Changes addressed
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20070: SPARK-22896 Improvement in String interpolation
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/20070#discussion_r158744300
--- Diff: examples/src/main/scala/org/apache/spark/examples/ml/QuantileDiscretizerExample.scala ---
@@ -31,12 +31,11 @@ object QuantileDiscretizerExample {
// $example on$
val data = Array((0, 18.0), (1, 19.0), (2, 8.0), (3, 5.0), (4, 2.2))
- val df = spark.createDataFrame(data).toDF("id", "hour")
+ val df = spark.createDataFrame(data).toDF("id", "hour").repartition(1)
--- End diff --
Although it looks weird, I think the author intended the repartition(1) to not appear in the body of the example that's copied into the docs. I wouldn't change this.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20070: SPARK-22896 Improvement in String interpolation
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20070
**[Test build #4029 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4029/testReport)** for PR 20070 at commit [`70ce734`](https://github.com/apache/spark/commit/70ce734681b0a6d7e833af35c6303bfc6c1aeac2).
* This patch **fails to build**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20070: SPARK-22896 Improvement in String interpolation
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20070
**[Test build #4031 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4031/testReport)** for PR 20070 at commit [`0321faf`](https://github.com/apache/spark/commit/0321faf054a94d7fc83b2a300529da2739813840).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20070: SPARK-22896 Improvement in String interpolation
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/20070#discussion_r158945588
--- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/AssociationRulesExample.scala ---
@@ -42,14 +42,13 @@ object AssociationRulesExample {
val results = ar.run(freqItemsets)
results.collect().foreach { rule =>
- println("[" + rule.antecedent.mkString(",")
- + "=>"
- + rule.consequent.mkString(",") + "]," + rule.confidence)
+ println(s"[${rule.antecedent.mkString(",")}=>${rule.consequent.mkString(",")} ]" +
+ s" ${rule.confidence}")
}
// $example off$
sc.stop()
}
}
-// scalastyle:on println
+// scalastyle:on println
--- End diff --
It didn't like the lack of newline at the end of this file
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20070: SPARK-22896 Improvement in String interpolation
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/20070#discussion_r159129797
--- Diff: examples/src/main/scala/org/apache/spark/examples/ml/QuantileDiscretizerExample.scala ---
@@ -45,7 +45,7 @@ object QuantileDiscretizerExample {
.setNumBuckets(3)
val result = discretizer.fit(df).transform(df)
- result.show()
+ result.show(false)
--- End diff --
One more question - is it necessary to make this not truncate?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20070: SPARK-22896 Improvement in String interpolation
Posted by chetkhatri <gi...@git.apache.org>.
Github user chetkhatri commented on a diff in the pull request:
https://github.com/apache/spark/pull/20070#discussion_r159084968
--- Diff: examples/src/main/scala/org/apache/spark/examples/ml/VectorIndexerExample.scala ---
@@ -41,8 +41,8 @@ object VectorIndexerExample {
val indexerModel = indexer.fit(data)
val categoricalFeatures: Set[Int] = indexerModel.categoryMaps.keys.toSet
- println(s"Chose ${categoricalFeatures.size} categorical features: " +
- categoricalFeatures.mkString(", "))
+ println(s"Chose ${categoricalFeatures.size} " +
+ s"categorical features: {$categoricalFeatures.mkString(", ")}")
--- End diff --
I did fixed this. Can you please give me steps as a check list before commit for test.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20070: SPARK-22896 Improvement in String interpolation
Posted by chetkhatri <gi...@git.apache.org>.
Github user chetkhatri commented on the issue:
https://github.com/apache/spark/pull/20070
@srowen Absolutely correct, this all in one shot. I did changes in all.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20070: SPARK-22896 Improvement in String interpolation
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/20070#discussion_r159143833
--- Diff: examples/src/main/scala/org/apache/spark/examples/ml/QuantileDiscretizerExample.scala ---
@@ -45,7 +45,7 @@ object QuantileDiscretizerExample {
.setNumBuckets(3)
val result = discretizer.fit(df).transform(df)
- result.show()
+ result.show(false)
--- End diff --
Which other examples? most do not set this, and the Java equivalent doesn't either. If there's a good reason that the output needs to be untruncated, that's fine, just also change the Java example.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20070: SPARK-22896 Improvement in String interpolation
Posted by chetkhatri <gi...@git.apache.org>.
Github user chetkhatri commented on a diff in the pull request:
https://github.com/apache/spark/pull/20070#discussion_r158976155
--- Diff: examples/src/main/scala/org/apache/spark/examples/ml/CorrelationExample.scala ---
@@ -51,10 +51,10 @@ object CorrelationExample {
val df = data.map(Tuple1.apply).toDF("features")
val Row(coeff1: Matrix) = Correlation.corr(df, "features").head
- println("Pearson correlation matrix:\n" + coeff1.toString)
+ println(s"Pearson correlation matrix:\n ${coeff1.toString}")
--- End diff --
Addressed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20070: SPARK-22896 Improvement in String interpolation
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/20070
Merged to master
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20070: SPARK-22896 Improvement in String interpolation
Posted by chetkhatri <gi...@git.apache.org>.
Github user chetkhatri commented on the issue:
https://github.com/apache/spark/pull/20070
@srowen I rechecked all scala examples and this is commulative PR for the same.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20070: SPARK-22896 Improvement in String interpolation
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20070
**[Test build #4031 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4031/testReport)** for PR 20070 at commit [`0321faf`](https://github.com/apache/spark/commit/0321faf054a94d7fc83b2a300529da2739813840).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20070: SPARK-22896 Improvement in String interpolation
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20070
**[Test build #4029 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4029/testReport)** for PR 20070 at commit [`70ce734`](https://github.com/apache/spark/commit/70ce734681b0a6d7e833af35c6303bfc6c1aeac2).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20070: SPARK-22896 Improvement in String interpolation
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/20070#discussion_r158744186
--- Diff: examples/src/main/scala/org/apache/spark/examples/graphx/Analytics.scala ---
@@ -145,9 +145,9 @@ object Analytics extends Logging {
// TriangleCount requires the graph to be partitioned
.partitionBy(partitionStrategy.getOrElse(RandomVertexCut)).cache()
val triangles = TriangleCount.run(graph)
- println("Triangles: " + triangles.vertices.map {
+ println(s"Triangles: ${triangles.vertices.map {
--- End diff --
This is probably more readable if the expression is stored in a local val first. Anything nontrivial like this gets hard to parse in an interpolated string. While we're here, worth fixing up.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20070: SPARK-22896 Improvement in String interpolation
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/20070#discussion_r159129807
--- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/HypothesisTestingExample.scala ---
@@ -68,7 +68,7 @@ object HypothesisTestingExample {
// against the label.
val featureTestResults: Array[ChiSqTestResult] = Statistics.chiSqTest(obs)
featureTestResults.zipWithIndex.foreach { case (k, v) =>
- println("Column " + (v + 1).toString + ":")
+ println(s"Column ${(v + 1).toString} :")
--- End diff --
.toString is redundant here and elsewhere with interpolation. I think that should be simplified.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20070: SPARK-22896 Improvement in String interpolation
Posted by chetkhatri <gi...@git.apache.org>.
Github user chetkhatri commented on the issue:
https://github.com/apache/spark/pull/20070
In scala ? I don't think so. I am re-iterating and doing double check.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20070: SPARK-22896 Improvement in String interpolation
Posted by chetkhatri <gi...@git.apache.org>.
Github user chetkhatri commented on the issue:
https://github.com/apache/spark/pull/20070
@srowen please do re-run the build.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20070: SPARK-22896 Improvement in String interpolation
Posted by chetkhatri <gi...@git.apache.org>.
Github user chetkhatri commented on the issue:
https://github.com/apache/spark/pull/20070
@srowen Okey. current status looks good
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20070: SPARK-22896 Improvement in String interpolation
Posted by chetkhatri <gi...@git.apache.org>.
Github user chetkhatri commented on the issue:
https://github.com/apache/spark/pull/20070
You're correct - I missed other packages. I will re-confirm soon. Thanks.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20070: SPARK-22896 Improvement in String interpolation
Posted by chetkhatri <gi...@git.apache.org>.
Github user chetkhatri commented on a diff in the pull request:
https://github.com/apache/spark/pull/20070#discussion_r159134529
--- Diff: examples/src/main/scala/org/apache/spark/examples/mllib/LatentDirichletAllocationExample.scala ---
@@ -42,11 +42,11 @@ object LatentDirichletAllocationExample {
val ldaModel = new LDA().setK(3).run(corpus)
// Output topics. Each is a distribution over words (matching word count vectors)
- println("Learned topics (as distributions over vocab of " + ldaModel.vocabSize + " words):")
+ println(s"Learned topics (as distributions over vocab of ${ldaModel.vocabSize} words):")
val topics = ldaModel.topicsMatrix
for (topic <- Range(0, 3)) {
- print("Topic " + topic + ":")
- for (word <- Range(0, ldaModel.vocabSize)) { print(" " + topics(word, topic)); }
+ print(s"Topic $topic :")
+ for (word <- Range(0, ldaModel.vocabSize)) { print(s" ${topics(word, topic)}") }
--- End diff --
@srowen Thanks for suggestion, it has been addressed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20070: SPARK-22896 Improvement in String interpolation
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/20070#discussion_r159062426
--- Diff: examples/src/main/scala/org/apache/spark/examples/ml/VectorIndexerExample.scala ---
@@ -41,8 +41,8 @@ object VectorIndexerExample {
val indexerModel = indexer.fit(data)
val categoricalFeatures: Set[Int] = indexerModel.categoryMaps.keys.toSet
- println(s"Chose ${categoricalFeatures.size} categorical features: " +
- categoricalFeatures.mkString(", "))
+ println(s"Chose ${categoricalFeatures.size} " +
+ s"categorical features: {$categoricalFeatures.mkString(", ")}")
--- End diff --
Typo here. Please make sure build/test/style passes locally before pushing again, or else this takes a lot of work to review.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org