You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by GitBox <gi...@apache.org> on 2021/05/29 11:16:21 UTC
[GitHub] [zeppelin] nilroy opened a new pull request #4127: Modified the interpreter.sh script to be able to parse SPARK_SUBMIT_OPTIONS
nilroy opened a new pull request #4127:
URL: https://github.com/apache/zeppelin/pull/4127
### What is this PR for?
To be able to parse SPARK_SUBMIT_OPTIONS which has whitespace in between like
```
export SPARK_SUBMIT_OPTIONS="$SPARK_SUBMIT_OPTIONS --conf 'spark.driver.extraJavaOptions=-Dcom.example.env=production -Dcom.example.role=zeppelin'"
```
### What type of PR is it?
Bug Fix
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-5397
### How should this be tested?
Its mentioned in the PR
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? Not sure
* Does this needs documentation? No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] nilroy edited a comment on pull request #4127: ZEPPELIN-5397 : Modified the interpreter.sh script to be able to parse SPARK_SUBMIT_OPTIONS
Posted by GitBox <gi...@apache.org>.
nilroy edited a comment on pull request #4127:
URL: https://github.com/apache/zeppelin/pull/4127#issuecomment-878844415
@Reamer @zjffdu
I did update as you suggested. However there are 3 other places where same IFS problem might show up:
- ZEPPELIN_FLINK_YARN_APPLICATION
- JAVA_INTP_OPTS
- ZEPPELIN_INTP_MEM
However I am not sure if that they uses similar pattern as SPARK_SUBMIT_OPTION
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] zjffdu commented on pull request #4127: ZEPPELIN-5397 : Modified the interpreter.sh script to be able to parse SPARK_SUBMIT_OPTIONS
Posted by GitBox <gi...@apache.org>.
zjffdu commented on pull request #4127:
URL: https://github.com/apache/zeppelin/pull/4127#issuecomment-891694553
@nilroy I mean the spark interpreter setting page, you can set all the spark configuration here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] zjffdu commented on pull request #4127: ZEPPELIN-5397 : Modified the interpreter.sh script to be able to parse SPARK_SUBMIT_OPTIONS
Posted by GitBox <gi...@apache.org>.
zjffdu commented on pull request #4127:
URL: https://github.com/apache/zeppelin/pull/4127#issuecomment-885484121
@Reamer CI is failed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] zjffdu commented on pull request #4127: ZEPPELIN-5397 : Modified the interpreter.sh script to be able to parse SPARK_SUBMIT_OPTIONS
Posted by GitBox <gi...@apache.org>.
zjffdu commented on pull request #4127:
URL: https://github.com/apache/zeppelin/pull/4127#issuecomment-878129113
> export SPARK_SUBMIT_OPTIONS="$SPARK_SUBMIT_OPTIONS --conf 'spark.driver.extraJavaOptions=-Dcom.example.env=production -Dcom.example.role=zeppelin' --conf mytest=myvalue"
>
> readarray -td '' SPARK_SUBMIT_OPTIONS_ARRAY < <(awk '{ gsub(/--conf/,"\0--conf"); print; }' <<<"${SPARK_SUBMIT_OPTIONS}--conf"); unset 'SPARK_SUBMIT_OPTIONS_ARRAY[-1]'
> declare -p SPARK_SUBMIT_OPTIONS_ARRAY;
@Reamer This approach LGTM, we need to do the same thing on `ZEPPELIN_SPARK_CONF`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] zjffdu commented on pull request #4127: ZEPPELIN-5397 : Modified the interpreter.sh script to be able to parse SPARK_SUBMIT_OPTIONS
Posted by GitBox <gi...@apache.org>.
zjffdu commented on pull request #4127:
URL: https://github.com/apache/zeppelin/pull/4127#issuecomment-887221869
@nilroy @Reamer The major problem of this PR is that `readarray` is not supported in mac, this would cause user unable to use zeppelin directly on mac. IIUC, this PR just use `|` as separator which is this PR does in java side. https://github.com/apache/zeppelin/pull/4173 This PR would work both in mac and linux.
Regarding `SPARK_SUBMIT_OPTIONS`, personally I think we should deprecate this option, we should ask user always set spark interpreter in interpreter setting which seems more consistent.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] nilroy commented on pull request #4127: ZEPPELIN-5397 : Modified the interpreter.sh script to be able to parse SPARK_SUBMIT_OPTIONS
Posted by GitBox <gi...@apache.org>.
nilroy commented on pull request #4127:
URL: https://github.com/apache/zeppelin/pull/4127#issuecomment-884787985
@Reamer Thank you. I shall leave it to you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] Reamer commented on pull request #4127: ZEPPELIN-5397 : Modified the interpreter.sh script to be able to parse SPARK_SUBMIT_OPTIONS
Posted by GitBox <gi...@apache.org>.
Reamer commented on pull request #4127:
URL: https://github.com/apache/zeppelin/pull/4127#issuecomment-853941348
I think an empty delimiter is wrong, possibly something like "--conf" would be better.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] zjffdu commented on pull request #4127: ZEPPELIN-5397 : Modified the interpreter.sh script to be able to parse SPARK_SUBMIT_OPTIONS
Posted by GitBox <gi...@apache.org>.
zjffdu commented on pull request #4127:
URL: https://github.com/apache/zeppelin/pull/4127#issuecomment-879460339
@nilroy Have you tried this PR to run spark yarn-cluster ? I suspect this PR cause the spark-submit fail to launch spark app.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] nilroy commented on pull request #4127: ZEPPELIN-5397 : Modified the interpreter.sh script to be able to parse SPARK_SUBMIT_OPTIONS
Posted by GitBox <gi...@apache.org>.
nilroy commented on pull request #4127:
URL: https://github.com/apache/zeppelin/pull/4127#issuecomment-891683323
@zjffdu @Reamer Personally I am not going to use the SPARK_SUBMIT_OPTIONS in my project after we hit this issue in the first place. We are already setting all spark configurations in spark-defaults.conf. So I am ok with removing that option completely. Regarding the interim solution #4173 I am perfectly ok with it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] nilroy commented on pull request #4127: ZEPPELIN-5397 : Modified the interpreter.sh script to be able to parse SPARK_SUBMIT_OPTIONS
Posted by GitBox <gi...@apache.org>.
nilroy commented on pull request #4127:
URL: https://github.com/apache/zeppelin/pull/4127#issuecomment-891692494
@zjffdu Could you please tell me the config file where I can set zeppelin spark interpreter setting? Is it the interpreter.default.json? I am a bit lost with the documentation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] Reamer commented on pull request #4127: ZEPPELIN-5397 : Modified the interpreter.sh script to be able to parse SPARK_SUBMIT_OPTIONS
Posted by GitBox <gi...@apache.org>.
Reamer commented on pull request #4127:
URL: https://github.com/apache/zeppelin/pull/4127#issuecomment-884218933
Hi @nilroy
I hope to have time to fix this. For the next two weeks I am on vacation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] nilroy commented on pull request #4127: ZEPPELIN-5397 : Modified the interpreter.sh script to be able to parse SPARK_SUBMIT_OPTIONS
Posted by GitBox <gi...@apache.org>.
nilroy commented on pull request #4127:
URL: https://github.com/apache/zeppelin/pull/4127#issuecomment-878844415
@Reamer @zjffdu
I did update as you suggested. However there are 3 other places where same IFS problem might show up:
- ZEPPELIN_FLINK_YARN_APPLICATION
- JAVA_INTP_OPTS
- ZEPPELIN_INTP_MEM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] Reamer commented on pull request #4127: ZEPPELIN-5397 : Modified the interpreter.sh script to be able to parse SPARK_SUBMIT_OPTIONS
Posted by GitBox <gi...@apache.org>.
Reamer commented on pull request #4127:
URL: https://github.com/apache/zeppelin/pull/4127#issuecomment-864825992
@nilroy I tried around a bit and found this [great explanation on Stackoverflow](https://stackoverflow.com/a/45201229)
I think the following solution can be used in our case. Tested at https://www.tutorialspoint.com/execute_bash_online.php
```
export SPARK_SUBMIT_OPTIONS="$SPARK_SUBMIT_OPTIONS --conf 'spark.driver.extraJavaOptions=-Dcom.example.env=production -Dcom.example.role=zeppelin' --conf mytest=myvalue"
readarray -td '' SPARK_SUBMIT_OPTIONS_ARRAY < <(awk '{ gsub(/--conf/,"\0--conf"); print; }' <<<"${SPARK_SUBMIT_OPTIONS}--conf"); unset 'SPARK_SUBMIT_OPTIONS_ARRAY[-1]'
declare -p SPARK_SUBMIT_OPTIONS_ARRAY;
```
What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] Reamer commented on pull request #4127: ZEPPELIN-5397 : Modified the interpreter.sh script to be able to parse SPARK_SUBMIT_OPTIONS
Posted by GitBox <gi...@apache.org>.
Reamer commented on pull request #4127:
URL: https://github.com/apache/zeppelin/pull/4127#issuecomment-853941348
I think an empty delimiter is wrong, possibly something like "--conf" would be better.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] Reamer commented on pull request #4127: ZEPPELIN-5397 : Modified the interpreter.sh script to be able to parse SPARK_SUBMIT_OPTIONS
Posted by GitBox <gi...@apache.org>.
Reamer commented on pull request #4127:
URL: https://github.com/apache/zeppelin/pull/4127#issuecomment-879604345
I see the following error quite often.
```
2021-07-13T05:53:46.8447034Z SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2021-07-13T05:53:46.8448574Z ##[error]Exception in thread "main" java.lang.NumberFormatException: For input string: "--conf spark.cores.max=2 "
2021-07-13T05:53:46.8450991Z at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
2021-07-13T05:53:46.8452128Z at java.lang.Integer.parseInt(Integer.java:580)
2021-07-13T05:53:46.8452816Z at java.lang.Integer.parseInt(Integer.java:615)
```
We missed the trim.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] zjffdu commented on pull request #4127: ZEPPELIN-5397 : Modified the interpreter.sh script to be able to parse SPARK_SUBMIT_OPTIONS
Posted by GitBox <gi...@apache.org>.
zjffdu commented on pull request #4127:
URL: https://github.com/apache/zeppelin/pull/4127#issuecomment-879197355
@nilroy CI is failed, https://github.com/apache/zeppelin/pull/4127/checks?check_run_id=3053525638
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] zjffdu commented on pull request #4127: ZEPPELIN-5397 : Modified the interpreter.sh script to be able to parse SPARK_SUBMIT_OPTIONS
Posted by GitBox <gi...@apache.org>.
zjffdu commented on pull request #4127:
URL: https://github.com/apache/zeppelin/pull/4127#issuecomment-886168602
@nilroy @Reamer I created another PR to fix the CI https://github.com/apache/zeppelin/pull/4187, it would be nice to have this merged in the next release.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] zjffdu commented on pull request #4127: ZEPPELIN-5397 : Modified the interpreter.sh script to be able to parse SPARK_SUBMIT_OPTIONS
Posted by GitBox <gi...@apache.org>.
zjffdu commented on pull request #4127:
URL: https://github.com/apache/zeppelin/pull/4127#issuecomment-891688607
Thanks for your feedback @nilroy Besides setting it in `spark-defaults.conf`, you can also set it in zeppelin's spark interpreter setting. Because setting it in `spark-defaults.conf` would affect all the spark application, while setting it in zeppelin's spark interpreter setting would only affect the spark application launched in zeppelin.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] Reamer commented on pull request #4127: ZEPPELIN-5397 : Modified the interpreter.sh script to be able to parse SPARK_SUBMIT_OPTIONS
Posted by GitBox <gi...@apache.org>.
Reamer commented on pull request #4127:
URL: https://github.com/apache/zeppelin/pull/4127#issuecomment-895027038
Closed in favor of #4173
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] nilroy edited a comment on pull request #4127: ZEPPELIN-5397 : Modified the interpreter.sh script to be able to parse SPARK_SUBMIT_OPTIONS
Posted by GitBox <gi...@apache.org>.
nilroy edited a comment on pull request #4127:
URL: https://github.com/apache/zeppelin/pull/4127#issuecomment-878844415
@Reamer @zjffdu
I did update as you suggested. However there are 3 other places where same IFS problem might show up:
- ZEPPELIN_FLINK_YARN_APPLICATION
- JAVA_INTP_OPTS
- ZEPPELIN_INTP_MEM
Shall we change those also?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] Reamer closed pull request #4127: ZEPPELIN-5397 : Modified the interpreter.sh script to be able to parse SPARK_SUBMIT_OPTIONS
Posted by GitBox <gi...@apache.org>.
Reamer closed pull request #4127:
URL: https://github.com/apache/zeppelin/pull/4127
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] nilroy commented on pull request #4127: ZEPPELIN-5397 : Modified the interpreter.sh script to be able to parse SPARK_SUBMIT_OPTIONS
Posted by GitBox <gi...@apache.org>.
nilroy commented on pull request #4127:
URL: https://github.com/apache/zeppelin/pull/4127#issuecomment-879276069
@zjffdu I am having hard time understanding the failures. They all seem to be Java stack traces. Can you help me debug?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] nilroy commented on pull request #4127: ZEPPELIN-5397 : Modified the interpreter.sh script to be able to parse SPARK_SUBMIT_OPTIONS
Posted by GitBox <gi...@apache.org>.
nilroy commented on pull request #4127:
URL: https://github.com/apache/zeppelin/pull/4127#issuecomment-891697278
@zjffdu Got it. Thank you. Shall I close this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] Reamer commented on pull request #4127: ZEPPELIN-5397 : Modified the interpreter.sh script to be able to parse SPARK_SUBMIT_OPTIONS
Posted by GitBox <gi...@apache.org>.
Reamer commented on pull request #4127:
URL: https://github.com/apache/zeppelin/pull/4127#issuecomment-864825992
@nilroy I tried around a bit and found this [great explanation on Stackoverflow](https://stackoverflow.com/a/45201229)
I think the following solution can be used in our case. Tested at https://www.tutorialspoint.com/execute_bash_online.php
```
export SPARK_SUBMIT_OPTIONS="$SPARK_SUBMIT_OPTIONS --conf 'spark.driver.extraJavaOptions=-Dcom.example.env=production -Dcom.example.role=zeppelin' --conf mytest=myvalue"
readarray -td '' SPARK_SUBMIT_OPTIONS_ARRAY < <(awk '{ gsub(/--conf/,"\0--conf"); print; }' <<<"${SPARK_SUBMIT_OPTIONS}--conf"); unset 'SPARK_SUBMIT_OPTIONS_ARRAY[-1]'
declare -p SPARK_SUBMIT_OPTIONS_ARRAY;
```
What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] nilroy edited a comment on pull request #4127: ZEPPELIN-5397 : Modified the interpreter.sh script to be able to parse SPARK_SUBMIT_OPTIONS
Posted by GitBox <gi...@apache.org>.
nilroy edited a comment on pull request #4127:
URL: https://github.com/apache/zeppelin/pull/4127#issuecomment-883986583
@zjffdu @Reamer I did not tested running a spark-yarn cluster. I am quite tied up with my work and getting it hard to invest time. If anyone of you can take this thing from here I shall be more than happy. You can close this PR and create a new one for the same Jira issue. Or don't change anything at all. We already found the way around this problem which I described earlier in a comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] zjffdu commented on pull request #4127: ZEPPELIN-5397 : Modified the interpreter.sh script to be able to parse SPARK_SUBMIT_OPTIONS
Posted by GitBox <gi...@apache.org>.
zjffdu commented on pull request #4127:
URL: https://github.com/apache/zeppelin/pull/4127#issuecomment-883351876
@nilroy Any update ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] zjffdu commented on pull request #4127: ZEPPELIN-5397 : Modified the interpreter.sh script to be able to parse SPARK_SUBMIT_OPTIONS
Posted by GitBox <gi...@apache.org>.
zjffdu commented on pull request #4127:
URL: https://github.com/apache/zeppelin/pull/4127#issuecomment-891077363
@nilroy @Reamer If you don't mind, I will use https://github.com/apache/zeppelin/pull/4173 as short term solution for the coming release, and we can think about the long term solution in the next release, what do you think ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] nilroy edited a comment on pull request #4127: ZEPPELIN-5397 : Modified the interpreter.sh script to be able to parse SPARK_SUBMIT_OPTIONS
Posted by GitBox <gi...@apache.org>.
nilroy edited a comment on pull request #4127:
URL: https://github.com/apache/zeppelin/pull/4127#issuecomment-891692494
@zjffdu Could you please tell me the config file where I can set zeppelin spark interpreter setting? Is it the interpreter.default.json? I am a bit lost with the documentation.
In our environment we are using separate mesos cluster for zeppelin and spark is using the executors from mesos. So setting spark-defaults.conf works in our use case. But I am eager to learn other alternatives.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] nilroy commented on pull request #4127: ZEPPELIN-5397 : Modified the interpreter.sh script to be able to parse SPARK_SUBMIT_OPTIONS
Posted by GitBox <gi...@apache.org>.
nilroy commented on pull request #4127:
URL: https://github.com/apache/zeppelin/pull/4127#issuecomment-883986583
@zjffdu I did not tested running a spark-yarn cluster. I am quite tied up with my work and getting it hard to invest time. If anyone of you can take this thing from here I shall be more than happy. You can close this PR and create a new one for the same Jira issue. Or don't change anything at all. We already found the way around this problem which I described earlier in a comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org