You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by mantovani <gi...@git.apache.org> on 2018/01/31 19:49:20 UTC
[GitHub] spark pull request #20458: changed scala example from java "style" to scala
GitHub user mantovani opened a pull request:
https://github.com/apache/spark/pull/20458
changed scala example from java "style" to scala
## What changes were proposed in this pull request?
I changed an example wrote in Scala using Java "style" to Scala "style".
## How was this patch tested?
It's not necessary test, the language documentation should be enough. For the documentation propose.
http://www.scala-lang.org/api/2.9.2/scala/App.html
Please review http://spark.apache.org/contributing.html before opening a pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mantovani/spark master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20458.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20458
----
commit bc708cb164b7c7818f58da7a76ae10f4a2d712c9
Author: Daniel <da...@...>
Date: 2018-01-31T19:41:58Z
changed scala example from java "style" to scala
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20458: changed scala example from java "style" to scala
Posted by mantovani <gi...@git.apache.org>.
Github user mantovani commented on the issue:
https://github.com/apache/spark/pull/20458
I just changed to the right section.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20458: changed scala example from java "style" to scala
Posted by mantovani <gi...@git.apache.org>.
Github user mantovani commented on the issue:
https://github.com/apache/spark/pull/20458
I just tested with Scala 2.10.6 and worked perfect.
18/01/31 17:21:28 INFO DAGScheduler: Job 1 finished: count at
Bootstrap.scala:14, took 0.024446 s
DemoBug: rslt1 = 3 rslt2 = 3
On Wed, Jan 31, 2018 at 5:16 PM, Marcelo Vanzin <no...@github.com>
wrote:
> Did you try it with Scala 2.10? That's still supported by Spark.
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/spark/pull/20458#issuecomment-362089291>, or mute
> the thread
> <https://github.com/notifications/unsubscribe-auth/AAC5fSAOjXpf1HNiOCQhx_BUL4GxaC4gks5tQOYggaJpZM4R0i6_>
> .
>
--
--
Daniel de Oliveira Mantovani
Perl Evangelist/Data Hacker
+1 786 459 1341
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20458: changed scala example from java "style" to scala
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/20458
> It's not just "will" work, it's working.
https://issues.apache.org/jira/browse/SPARK-4170
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20458: changed scala example from java "style" to scala
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20458
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20458: changed scala example from java "style" to scala
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/20458
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20458: changed scala example from java "style" to scala
Posted by mantovani <gi...@git.apache.org>.
Github user mantovani commented on the issue:
https://github.com/apache/spark/pull/20458
Hello Marcel,
This is an old bug and was fixed long time ago, I just runned the following
example and got the correct result:
DemoBug: rslt1 = 3 rslt2 = 3
Doesn't make sense use Java traditional way "main(...)" just because 4
years ago the "Scala approach" had a bug. Sometimes we can't use feature
because of bugs, it's fine but we should move on when they are fixed.
On Wed, Jan 31, 2018 at 4:20 PM, Marcelo Vanzin <no...@github.com>
wrote:
> It's not just "will" work, it's working.
>
> https://issues.apache.org/jira/browse/SPARK-4170
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/spark/pull/20458#issuecomment-362074134>, or mute
> the thread
> <https://github.com/notifications/unsubscribe-auth/AAC5fS39XmaJETebqFaYm7SzKOtDk1KVks5tQNkWgaJpZM4R0i6_>
> .
>
--
--
Daniel de Oliveira Mantovani
Perl Evangelist/Data Hacker
+1 786 459 1341
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20458: changed scala example from java "style" to scala
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20458
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20458: changed scala example from java "style" to scala
Posted by mantovani <gi...@git.apache.org>.
Github user mantovani commented on the issue:
https://github.com/apache/spark/pull/20458
If I change all examples of the documentation will be accept ?
On Wed, Jan 31, 2018 at 6:41 PM, Sean Owen <no...@github.com> wrote:
> Yeah, that's what I was thinking of. Subclasses of App don't necessarily
> work in Spark (not just Scala). I don't know if that has changed as Spark
> and Scala moved forward. While that's an interesting item to investigate,
> here the question is changing one example to use App, while none of the
> others do and there's a reason not to. We don't want to do that.
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/spark/pull/20458#issuecomment-362108399>, or mute
> the thread
> <https://github.com/notifications/unsubscribe-auth/AAC5fXruoTYCd0hMD4wsXtMSNUYB-pz5ks5tQPohgaJpZM4R0i6_>
> .
>
--
--
Daniel de Oliveira Mantovani
Perl Evangelist/Data Hacker
+1 786 459 1341
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20458: changed scala example from java "style" to scala
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/20458
The point is that it won't always work in Spark, so, certainly not unless you test them. I just don't think it's worth it, especially as it's setting an expectation as an example that it is the best way to do it.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20458: changed scala example from java "style" to scala
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/20458
Yeah, that's what I was thinking of. Subclasses of `App` don't necessarily work in Spark (not just Scala). I don't know if that has changed as Spark and Scala moved forward. While that's an interesting item to investigate, here the question is changing one example to use `App`, while none of the others do and there's a reason not to. We don't want to do that.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20458: changed scala example from java "style" to scala
Posted by mantovani <gi...@git.apache.org>.
Github user mantovani commented on the issue:
https://github.com/apache/spark/pull/20458
Sean, I already tested. Can you please be more specific how should be the
tests ? I just tested on Spark 2.1.1 and Scala 2.11 and 2.10.
On Wed, Jan 31, 2018 at 19:19 Sean Owen <no...@github.com> wrote:
> The point is that it won't always work in Spark, so, certainly not unless
> you test them. I just don't think it's worth it, especially as it's setting
> an expectation as an example that it is the best way to do it.
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/spark/pull/20458#issuecomment-362115474>, or mute
> the thread
> <https://github.com/notifications/unsubscribe-auth/AAC5fQJd6YYF8AI0tijFEkzDKExgiABUks5tQQL_gaJpZM4R0i6_>
> .
>
--
--
Daniel de Oliveira Mantovani
Perl Evangelist/Data Hacker
+1 786 459 1341
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20458: changed scala example from java "style" to scala
Posted by mgaido91 <gi...@git.apache.org>.
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/20458
the example is written in Java, not in Scala. I think you can close this PR, thanks.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20458: changed scala example from java "style" to scala
Posted by mantovani <gi...@git.apache.org>.
Github user mantovani commented on a diff in the pull request:
https://github.com/apache/spark/pull/20458#discussion_r165180039
--- Diff: docs/quick-start.md ---
@@ -299,19 +299,17 @@ We'll create a very simple Spark application, `SimpleApp.java`:
import org.apache.spark.sql.SparkSession;
import org.apache.spark.sql.Dataset;
-public class SimpleApp {
- public static void main(String[] args) {
- String logFile = "YOUR_SPARK_HOME/README.md"; // Should be some file on your system
- SparkSession spark = SparkSession.builder().appName("Simple Application").getOrCreate();
- Dataset<String> logData = spark.read().textFile(logFile).cache();
+public class SimpleApp extends App {
--- End diff --
Sorry, I updated the wrong section.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20458: changed scala example from java "style" to scala
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20458
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20458: changed scala example from java "style" to scala
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/20458
IIRC extending App won't work, or didn't. The example is already correct.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20458: changed scala example from java "style" to scala
Posted by mantovani <gi...@git.apache.org>.
Github user mantovani commented on the issue:
https://github.com/apache/spark/pull/20458
It's not just "will" work, _it's working_. I never said that it wasn't working, the Scala example was wrote using Java "style" not Scala "style" which . Which you don't create a main method but extends App.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20458: changed scala example from java "style" to scala
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/20458#discussion_r165178509
--- Diff: docs/quick-start.md ---
@@ -299,19 +299,17 @@ We'll create a very simple Spark application, `SimpleApp.java`:
import org.apache.spark.sql.SparkSession;
import org.apache.spark.sql.Dataset;
-public class SimpleApp {
- public static void main(String[] args) {
- String logFile = "YOUR_SPARK_HOME/README.md"; // Should be some file on your system
- SparkSession spark = SparkSession.builder().appName("Simple Application").getOrCreate();
- Dataset<String> logData = spark.read().textFile(logFile).cache();
+public class SimpleApp extends App {
--- End diff --
Yes, this isn't valid Java, and it's supposed to be Java. It was correct before.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20458: changed scala example from java "style" to scala
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/20458
Did you try it with Scala 2.10? That's still supported by Spark.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org