You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by gaborgsomogyi <gi...@git.apache.org> on 2018/05/29 15:58:11 UTC
[GitHub] spark pull request #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit exec...
GitHub user gaborgsomogyi opened a pull request:
https://github.com/apache/spark/pull/21450
[SPARK-24319][SPARK SUBMIT] Fix spark-submit execution where no main class is required.
## What changes were proposed in this pull request?
With [PR 20925](https://github.com/apache/spark/pull/20925) now it's not possible to execute the following commands:
* run-example --help
* run-example --version
* run-example --usage-error
* run-example --status ...
* run-example --kill ...
In this PR the execution will be allowed for the mentioned commands.
## How was this patch tested?
Existing unit tests extended + additional written.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gaborgsomogyi/spark SPARK-24319
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21450.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21450
----
commit a69850b6fdcbe2e234e70a597d9ad6beae6a6937
Author: Gabor Somogyi <ga...@...>
Date: 2018-05-29T15:38:56Z
[SPARK-24319][SPARK SUBMIT] Fix spark-submit execution where no main class is required.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21450
**[Test build #91390 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91390/testReport)** for PR 21450 at commit [`12a5145`](https://github.com/apache/spark/commit/12a5145464dc8429ec9b120afb86a0b984deed93).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit exec...
Posted by gaborgsomogyi <gi...@git.apache.org>.
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/21450#discussion_r195016477
--- Diff: launcher/src/main/java/org/apache/spark/launcher/SparkSubmitCommandBuilder.java ---
@@ -138,25 +139,30 @@
case RUN_EXAMPLE:
isExample = true;
+ appResource = SparkLauncher.NO_RESOURCE;
submitArgs = args.subList(1, args.size());
}
this.isExample = isExample;
- OptionParser parser = new OptionParser(true);
- parser.parse(submitArgs);
- this.requiresAppResource = parser.requiresAppResource;
+ if (!submitArgs.isEmpty()) {
--- End diff --
Removed as it leads to the same result.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit exec...
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21450#discussion_r191851612
--- Diff: launcher/src/main/java/org/apache/spark/launcher/SparkSubmitCommandBuilder.java ---
@@ -229,7 +238,7 @@
args.add(join(",", pyFiles));
}
- if (isExample) {
+ if (isExample && requiresMainClass) {
--- End diff --
In what situation will `requiresAppResource` be different from `requiresMainClass` at this point?
It seems to me they're basically the same. The only point in your code where they differ is in the no-arg constructor, which is not used by the `run-example` script.
Seems to me like you could rename the existing variable to `isSpecialCommand` or something and use it instead for both cases.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21450
**[Test build #91598 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91598/testReport)** for PR 21450 at commit [`12a5145`](https://github.com/apache/spark/commit/12a5145464dc8429ec9b120afb86a0b984deed93).
* This patch **fails Java style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21450
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91262/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by gaborgsomogyi <gi...@git.apache.org>.
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/21450
The issue seems unrelated.
```
[error] (kubernetes-integration-tests/*:checkstyle) java.io.FileNotFoundException: checkstyle-config.xml (No such file or directory)
```
Trying to rebase the changes to the latest master.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit exec...
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21450#discussion_r194916287
--- Diff: launcher/src/test/java/org/apache/spark/launcher/SparkSubmitCommandBuilderSuite.java ---
@@ -190,6 +197,36 @@ public void testSparkRShell() throws Exception {
env.get("SPARKR_SUBMIT_ARGS"));
}
+ @Test(expected = IllegalArgumentException.class)
+ public void testExamplesRunnerNoArg() throws Exception {
+ List<String> sparkSubmitArgs = Arrays.asList(SparkSubmitCommandBuilder.RUN_EXAMPLE);
+ Map<String, String> env = new HashMap<>();
+ buildCommand(sparkSubmitArgs, env);
+ }
+
+ @Test
+ public void testExamplesRunnerNoMainClass() throws Exception {
+ testCLIOpts(SparkSubmitCommandBuilder.RUN_EXAMPLE, parser.HELP, null);
+ testCLIOpts(SparkSubmitCommandBuilder.RUN_EXAMPLE, parser.USAGE_ERROR, null);
+ testCLIOpts(SparkSubmitCommandBuilder.RUN_EXAMPLE, parser.VERSION, null);
+ }
+
+ @Rule
+ public ExpectedException testExamplesRunnerWithMasterNoMainClassEx = ExpectedException.none();
+
+ @Test
+ public void testExamplesRunnerWithMasterNoMainClass() throws Exception {
+ testExamplesRunnerWithMasterNoMainClassEx.expect(IllegalArgumentException.class);
--- End diff --
I have a slight preference for a `try...catch` in this case, which would also be less code, but either is fine. But I'd pick a shorter and more generic name for the rule (since it could be reused in other tests if needed later).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21450
ok tk test
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit exec...
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21450#discussion_r191851844
--- Diff: launcher/src/test/java/org/apache/spark/launcher/SparkSubmitCommandBuilderSuite.java ---
@@ -344,8 +365,15 @@ private SparkSubmitCommandBuilder newCommandBuilder(List<String> args) {
return newCommandBuilder(args).buildCommand(env);
}
- private void testCLIOpts(String opt) throws Exception {
- List<String> helpArgs = Arrays.asList(opt, "driver-20160531171222-0000");
+ private void testCLIOpts(String appResource, String opt, List<String> params) throws Exception {
+ List<String> helpArgs = new ArrayList<>();
--- End diff --
Since you're changing this, probably better to rename the variable since these are not help arguments.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21450
**[Test build #91698 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91698/testReport)** for PR 21450 at commit [`b03e3de`](https://github.com/apache/spark/commit/b03e3de9b326a8cf9061125e0f22bde2a12bf30f).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21450
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit exec...
Posted by BryanCutler <gi...@git.apache.org>.
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/21450#discussion_r191928359
--- Diff: launcher/src/test/java/org/apache/spark/launcher/SparkSubmitCommandBuilderSuite.java ---
@@ -190,6 +194,23 @@ public void testSparkRShell() throws Exception {
env.get("SPARKR_SUBMIT_ARGS"));
}
+ @Test
+ public void testExamplesRunnerNoArg() throws Exception {
--- End diff --
When no args are given this is what I get as the output:
```
$ bin/run-example
Error: Must specify a primary resource (JAR or Python or R file)
Run with --help for usage help or --verbose for debug output
```
It doesn't really make sense asking for a primary resource when running an example, how about just printing usage?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21450
ok to test
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21450
**[Test build #91853 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91853/testReport)** for PR 21450 at commit [`5a737d7`](https://github.com/apache/spark/commit/5a737d7d9dc11ee84744f57700d4c0a1b13fb73b).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit exec...
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21450#discussion_r195165247
--- Diff: launcher/src/main/java/org/apache/spark/launcher/Main.java ---
@@ -101,6 +99,23 @@ public static void main(String[] argsArray) throws Exception {
}
}
+ /**
+ * Prepare spark commands with the appropriate command builder.
+ * If printLaunchCommand is set then the commands will be printed to the stderr.
+ */
+ private static List<String> buildCommand(
+ AbstractCommandBuilder builder,
--- End diff --
The style is to double indent parameter lists... when in doubt, look at the rest of the code, not the style guide.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21450
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91598/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21450
**[Test build #91267 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91267/testReport)** for PR 21450 at commit [`a69850b`](https://github.com/apache/spark/commit/a69850b6fdcbe2e234e70a597d9ad6beae6a6937).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21450
**[Test build #91853 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91853/testReport)** for PR 21450 at commit [`5a737d7`](https://github.com/apache/spark/commit/5a737d7d9dc11ee84744f57700d4c0a1b13fb73b).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit exec...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21450
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by gaborgsomogyi <gi...@git.apache.org>.
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/21450
cc @vanzin
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/21450
This doesn't seem to be addressing the issue reported in the bug. The exact same error happens with your patch:
```
$ ./bin/run-example
Exception in thread "main" java.lang.IllegalArgumentException: Missing application resource.
at org.apache.spark.launcher.CommandBuilderUtils.checkArgument(CommandBuilderUtils.java:241)
at org.apache.spark.launcher.SparkSubmitCommandBuilder.buildSparkSubmitArgs(SparkSubmitCommandBuilder.java:185)
at org.apache.spark.launcher.SparkSubmitCommandBuilder.buildSparkSubmitCommand(SparkSubmitCommandBuilder.java:300)
at org.apache.spark.launcher.SparkSubmitCommandBuilder.buildCommand(SparkSubmitCommandBuilder.java:166)
at org.apache.spark.launcher.Main.main(Main.java:86)
```
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21450
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit exec...
Posted by gaborgsomogyi <gi...@git.apache.org>.
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/21450#discussion_r192385957
--- Diff: launcher/src/test/java/org/apache/spark/launcher/SparkSubmitCommandBuilderSuite.java ---
@@ -344,8 +365,15 @@ private SparkSubmitCommandBuilder newCommandBuilder(List<String> args) {
return newCommandBuilder(args).buildCommand(env);
}
- private void testCLIOpts(String opt) throws Exception {
- List<String> helpArgs = Arrays.asList(opt, "driver-20160531171222-0000");
+ private void testCLIOpts(String appResource, String opt, List<String> params) throws Exception {
+ List<String> helpArgs = new ArrayList<>();
--- End diff --
Renamed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21450
**[Test build #91698 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91698/testReport)** for PR 21450 at commit [`b03e3de`](https://github.com/apache/spark/commit/b03e3de9b326a8cf9061125e0f22bde2a12bf30f).
* This patch **fails Java style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21450
**[Test build #91763 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91763/testReport)** for PR 21450 at commit [`8a8067e`](https://github.com/apache/spark/commit/8a8067ec89705fb8acaec82eecd94c2312aeea76).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21450
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21450
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21450
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21450
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21450
**[Test build #91700 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91700/testReport)** for PR 21450 at commit [`d5a9ca3`](https://github.com/apache/spark/commit/d5a9ca350ce3a3e498dec07b8cd6b4fcef626e8c).
* This patch **fails Java style tests**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* ` logInfo(s\"Using output committer class $`
* `public class JavaPowerIterationClusteringExample `
* `class GBTClassificationModel(TreeEnsembleModel, JavaClassificationModel, JavaMLWritable,`
* `class PowerIterationClustering(HasMaxIter, HasWeightCol, JavaParams, JavaMLReadable,`
* `class PrefixSpan(JavaParams):`
* `public class MaskExpressionsUtils `
* `case class ArrayRemove(left: Expression, right: Expression)`
* `trait MaskLike `
* `trait MaskLikeWithN extends MaskLike `
* `case class Mask(child: Expression, upper: String, lower: String, digit: String)`
* `case class MaskFirstN(`
* `case class MaskLastN(`
* `case class MaskShowFirstN(`
* `case class MaskShowLastN(`
* `case class MaskHash(child: Expression)`
* `abstract class FileFormatDataWriter(`
* `class EmptyDirectoryDataWriter(`
* `class SingleDirectoryDataWriter(`
* `class DynamicPartitionDataWriter(`
* `class WriteJobDescription(`
* `case class WriteTaskResult(commitMsg: TaskCommitMessage, summary: ExecutedWriteSummary)`
* `case class ExecutedWriteSummary(`
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21450
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91267/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21450
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91390/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21450
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91763/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21450
**[Test build #91700 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91700/testReport)** for PR 21450 at commit [`d5a9ca3`](https://github.com/apache/spark/commit/d5a9ca350ce3a3e498dec07b8cd6b4fcef626e8c).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit exec...
Posted by gaborgsomogyi <gi...@git.apache.org>.
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/21450#discussion_r195016532
--- Diff: launcher/src/main/java/org/apache/spark/launcher/Main.java ---
@@ -101,6 +99,22 @@ public static void main(String[] argsArray) throws Exception {
}
}
+ /**
+ * Prepare spark commands with the appropriate command builder.
+ * If printLaunchCommand is set then the commands will be printed to the stderr.
+ */
+ private static List<String> buildCommand(AbstractCommandBuilder builder,
--- End diff --
Fixed.
Just for the sake of my understanding what does `many` mean in the following? All?
`For Java code, Apache Spark follows Oracle’s Java code conventions. Many Scala guidelines below also apply to Java.`
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21450
**[Test build #91763 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91763/testReport)** for PR 21450 at commit [`8a8067e`](https://github.com/apache/spark/commit/8a8067ec89705fb8acaec82eecd94c2312aeea76).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21450
**[Test build #91267 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91267/testReport)** for PR 21450 at commit [`a69850b`](https://github.com/apache/spark/commit/a69850b6fdcbe2e234e70a597d9ad6beae6a6937).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21450
**[Test build #91262 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91262/testReport)** for PR 21450 at commit [`a69850b`](https://github.com/apache/spark/commit/a69850b6fdcbe2e234e70a597d9ad6beae6a6937).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/21450
This is still missing something.
```
$ ./bin/run-example --master local
Exception in thread "main" java.lang.IllegalArgumentException: Missing application resource.
at org.apache.spark.launcher.CommandBuilderUtils.checkArgument(CommandBuilderUtils.java:241)
at org.apache.spark.launcher.SparkSubmitCommandBuilder.buildSparkSubmitArgs(SparkSubmitCommandBuilder.java:189)
at org.apache.spark.launcher.SparkSubmitCommandBuilder.buildSparkSubmitCommand(SparkSubmitCommandBuilder.java:304)
at org.apache.spark.launcher.SparkSubmitCommandBuilder.buildCommand(SparkSubmitCommandBuilder.java:170)
at org.apache.spark.launcher.Main.main(Main.java:86)
```
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21450
**[Test build #91299 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91299/testReport)** for PR 21450 at commit [`9052dff`](https://github.com/apache/spark/commit/9052dff88709a02f63cbaca4bece38075a9ab402).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/21450
Looks good. "run-example --kill blah" is kinda weird, but not worth it to handle that case.
Merging to master.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21450
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit exec...
Posted by gaborgsomogyi <gi...@git.apache.org>.
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/21450#discussion_r192385912
--- Diff: launcher/src/main/java/org/apache/spark/launcher/SparkSubmitCommandBuilder.java ---
@@ -229,7 +238,7 @@
args.add(join(",", pyFiles));
}
- if (isExample) {
+ if (isExample && requiresMainClass) {
--- End diff --
Originally I was hesitating to merge them and loose the 2 different meaning but it's better. Added a small comment.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21450
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21450
**[Test build #91598 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91598/testReport)** for PR 21450 at commit [`12a5145`](https://github.com/apache/spark/commit/12a5145464dc8429ec9b120afb86a0b984deed93).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit exec...
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21450#discussion_r195144445
--- Diff: launcher/src/main/java/org/apache/spark/launcher/Main.java ---
@@ -101,6 +99,22 @@ public static void main(String[] argsArray) throws Exception {
}
}
+ /**
+ * Prepare spark commands with the appropriate command builder.
+ * If printLaunchCommand is set then the commands will be printed to the stderr.
+ */
+ private static List<String> buildCommand(AbstractCommandBuilder builder,
--- End diff --
Read that as "unless it cannot apply to Java at all, follow the Scala style".
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit exec...
Posted by gaborgsomogyi <gi...@git.apache.org>.
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/21450#discussion_r195016584
--- Diff: launcher/src/test/java/org/apache/spark/launcher/SparkSubmitCommandBuilderSuite.java ---
@@ -190,6 +197,36 @@ public void testSparkRShell() throws Exception {
env.get("SPARKR_SUBMIT_ARGS"));
}
+ @Test(expected = IllegalArgumentException.class)
+ public void testExamplesRunnerNoArg() throws Exception {
+ List<String> sparkSubmitArgs = Arrays.asList(SparkSubmitCommandBuilder.RUN_EXAMPLE);
+ Map<String, String> env = new HashMap<>();
+ buildCommand(sparkSubmitArgs, env);
+ }
+
+ @Test
+ public void testExamplesRunnerNoMainClass() throws Exception {
+ testCLIOpts(SparkSubmitCommandBuilder.RUN_EXAMPLE, parser.HELP, null);
+ testCLIOpts(SparkSubmitCommandBuilder.RUN_EXAMPLE, parser.USAGE_ERROR, null);
+ testCLIOpts(SparkSubmitCommandBuilder.RUN_EXAMPLE, parser.VERSION, null);
+ }
+
+ @Rule
+ public ExpectedException testExamplesRunnerWithMasterNoMainClassEx = ExpectedException.none();
+
+ @Test
+ public void testExamplesRunnerWithMasterNoMainClass() throws Exception {
+ testExamplesRunnerWithMasterNoMainClassEx.expect(IllegalArgumentException.class);
--- End diff --
Renamed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21450
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91700/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit exec...
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21450#discussion_r191929581
--- Diff: launcher/src/test/java/org/apache/spark/launcher/SparkSubmitCommandBuilderSuite.java ---
@@ -190,6 +194,23 @@ public void testSparkRShell() throws Exception {
env.get("SPARKR_SUBMIT_ARGS"));
}
+ @Test
+ public void testExamplesRunnerNoArg() throws Exception {
--- End diff --
Right, the output should make sense also.
When the launcher library detects an invalid command it should be running `spark-submit --usage-error`, which should print usage with the overridden description from the `run-example` script. That's done in `Main.java` when it catches an `IllegalArgumentException`.
Seems that's not exactly what's happening here.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by gaborgsomogyi <gi...@git.apache.org>.
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/21450
Updated the description to reflect the no arg case as well.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21450
**[Test build #91390 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91390/testReport)** for PR 21450 at commit [`12a5145`](https://github.com/apache/spark/commit/12a5145464dc8429ec9b120afb86a0b984deed93).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21450
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91698/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21450
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21450
**[Test build #91299 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91299/testReport)** for PR 21450 at commit [`9052dff`](https://github.com/apache/spark/commit/9052dff88709a02f63cbaca4bece38075a9ab402).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit exec...
Posted by gaborgsomogyi <gi...@git.apache.org>.
Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/21450#discussion_r192386271
--- Diff: launcher/src/test/java/org/apache/spark/launcher/SparkSubmitCommandBuilderSuite.java ---
@@ -190,6 +194,23 @@ public void testSparkRShell() throws Exception {
env.get("SPARKR_SUBMIT_ARGS"));
}
+ @Test
+ public void testExamplesRunnerNoArg() throws Exception {
--- End diff --
Now it prints out the usage.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21450
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91853/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by gaborgsomogyi <gi...@git.apache.org>.
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/21450
test this please
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21450
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21450
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit execution w...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21450
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91299/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit exec...
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21450#discussion_r194915218
--- Diff: launcher/src/main/java/org/apache/spark/launcher/SparkSubmitCommandBuilder.java ---
@@ -138,25 +139,30 @@
case RUN_EXAMPLE:
isExample = true;
+ appResource = SparkLauncher.NO_RESOURCE;
submitArgs = args.subList(1, args.size());
}
this.isExample = isExample;
- OptionParser parser = new OptionParser(true);
- parser.parse(submitArgs);
- this.requiresAppResource = parser.requiresAppResource;
+ if (!submitArgs.isEmpty()) {
--- End diff --
Do you need this check? `parser.isSpecialCommand` is initialized to `false`, so seems like it's already handled.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #21450: [SPARK-24319][SPARK SUBMIT] Fix spark-submit exec...
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21450#discussion_r194915296
--- Diff: launcher/src/main/java/org/apache/spark/launcher/Main.java ---
@@ -101,6 +99,22 @@ public static void main(String[] argsArray) throws Exception {
}
}
+ /**
+ * Prepare spark commands with the appropriate command builder.
+ * If printLaunchCommand is set then the commands will be printed to the stderr.
+ */
+ private static List<String> buildCommand(AbstractCommandBuilder builder,
--- End diff --
Please follow the Spark style for multi-line parameter lists.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org