You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by GitBox <gi...@apache.org> on 2022/01/29 02:27:34 UTC
[GitHub] [zeppelin] huage1994 opened a new pull request #4293: [ZEPPELIN-5639] Support node label in yarn interpreter launcher
huage1994 opened a new pull request #4293:
URL: https://github.com/apache/zeppelin/pull/4293
### What is this PR for?
Support node label in yarn interpreter launcher,
so that user can run interpreter in specific node.
### What type of PR is it?
[Feature]
### Todos
* [ ] - Task
### What is the Jira issue?
* Open an issue on Jira https://issues.apache.org/jira/browse/ZEPPELIN-5639
### How should this be tested?
CI passed
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] zjffdu commented on a change in pull request #4293: [ZEPPELIN-5639] Support node label in yarn interpreter launcher
Posted by GitBox <gi...@apache.org>.
zjffdu commented on a change in pull request #4293:
URL: https://github.com/apache/zeppelin/pull/4293#discussion_r794986734
##########
File path: zeppelin-plugins/launcher/yarn/src/main/java/org/apache/zeppelin/interpreter/launcher/YarnRemoteInterpreterProcess.java
##########
@@ -178,6 +178,10 @@ public void start(String userName) throws IOException {
this.appId = appResponse.getApplicationId();
ApplicationSubmissionContext appContext = yarnApplication.getApplicationSubmissionContext();
appContext = createApplicationSubmissionContext(appContext);
+ String YARN_NODE_LABEL_EXPRESSION = "yarn.node.label.expression";
Review comment:
It is better to name it as `zeppelin.interpreter.yarn.node.label.expression` to make it consistent with memory and core configuration (`zeppelin.interpreter.yarn.resource.memory` & `zeppelin.interpreter.yarn.resource.cores`).
Besides that, the document also needs to be updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] asfgit closed pull request #4293: [ZEPPELIN-5639] Support node label in yarn interpreter launcher
Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #4293:
URL: https://github.com/apache/zeppelin/pull/4293
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] zjffdu commented on pull request #4293: [ZEPPELIN-5639] Support node label in yarn interpreter launcher
Posted by GitBox <gi...@apache.org>.
zjffdu commented on pull request #4293:
URL: https://github.com/apache/zeppelin/pull/4293#issuecomment-1040433434
LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] huage1994 commented on pull request #4293: [ZEPPELIN-5639] Support node label in yarn interpreter launcher
Posted by GitBox <gi...@apache.org>.
huage1994 commented on pull request #4293:
URL: https://github.com/apache/zeppelin/pull/4293#issuecomment-1049551231
> @huage1994 Could you rebase to the latest master?
Yeah sure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] zjffdu commented on pull request #4293: [ZEPPELIN-5639] Support node label in yarn interpreter launcher
Posted by GitBox <gi...@apache.org>.
zjffdu commented on pull request #4293:
URL: https://github.com/apache/zeppelin/pull/4293#issuecomment-1040433434
LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] zjffdu commented on a change in pull request #4293: [ZEPPELIN-5639] Support node label in yarn interpreter launcher
Posted by GitBox <gi...@apache.org>.
zjffdu commented on a change in pull request #4293:
URL: https://github.com/apache/zeppelin/pull/4293#discussion_r795060042
##########
File path: zeppelin-plugins/launcher/yarn/src/main/java/org/apache/zeppelin/interpreter/launcher/YarnRemoteInterpreterProcess.java
##########
@@ -178,6 +178,10 @@ public void start(String userName) throws IOException {
this.appId = appResponse.getApplicationId();
ApplicationSubmissionContext appContext = yarnApplication.getApplicationSubmissionContext();
appContext = createApplicationSubmissionContext(appContext);
+ String YARN_NODE_LABEL_EXPRESSION = "zeppelin.interpreter.yarn.node.label.expression";
+ if(properties.containsKey(YARN_NODE_LABEL_EXPRESSION) && StringUtils.isNotEmpty(properties.getProperty(YARN_NODE_LABEL_EXPRESSION))) {
Review comment:
You can use `properties.getProperty(key, defaultValue)` to simplify it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] huage1994 commented on a change in pull request #4293: [ZEPPELIN-5639] Support node label in yarn interpreter launcher
Posted by GitBox <gi...@apache.org>.
huage1994 commented on a change in pull request #4293:
URL: https://github.com/apache/zeppelin/pull/4293#discussion_r795164757
##########
File path: zeppelin-plugins/launcher/yarn/src/main/java/org/apache/zeppelin/interpreter/launcher/YarnRemoteInterpreterProcess.java
##########
@@ -178,6 +178,10 @@ public void start(String userName) throws IOException {
this.appId = appResponse.getApplicationId();
ApplicationSubmissionContext appContext = yarnApplication.getApplicationSubmissionContext();
appContext = createApplicationSubmissionContext(appContext);
+ String YARN_NODE_LABEL_EXPRESSION = "zeppelin.interpreter.yarn.node.label.expression";
+ if(properties.containsKey(YARN_NODE_LABEL_EXPRESSION) && StringUtils.isNotEmpty(properties.getProperty(YARN_NODE_LABEL_EXPRESSION))) {
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] zjffdu commented on pull request #4293: [ZEPPELIN-5639] Support node label in yarn interpreter launcher
Posted by GitBox <gi...@apache.org>.
zjffdu commented on pull request #4293:
URL: https://github.com/apache/zeppelin/pull/4293#issuecomment-1048619548
@huage1994 Could you rebase to the latest master?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] huage1994 commented on a change in pull request #4293: [ZEPPELIN-5639] Support node label in yarn interpreter launcher
Posted by GitBox <gi...@apache.org>.
huage1994 commented on a change in pull request #4293:
URL: https://github.com/apache/zeppelin/pull/4293#discussion_r795164516
##########
File path: zeppelin-plugins/launcher/yarn/src/main/java/org/apache/zeppelin/interpreter/launcher/YarnRemoteInterpreterProcess.java
##########
@@ -178,6 +178,10 @@ public void start(String userName) throws IOException {
this.appId = appResponse.getApplicationId();
ApplicationSubmissionContext appContext = yarnApplication.getApplicationSubmissionContext();
appContext = createApplicationSubmissionContext(appContext);
+ String YARN_NODE_LABEL_EXPRESSION = "zeppelin.interpreter.yarn.node.label.expression";
+ if(properties.containsKey(YARN_NODE_LABEL_EXPRESSION) && StringUtils.isNotEmpty(properties.getProperty(YARN_NODE_LABEL_EXPRESSION))) {
Review comment:
> You can use `properties.getProperty(key, defaultValue)` to simplify it
Yes, it's a good idea
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] huage1994 commented on a change in pull request #4293: [ZEPPELIN-5639] Support node label in yarn interpreter launcher
Posted by GitBox <gi...@apache.org>.
huage1994 commented on a change in pull request #4293:
URL: https://github.com/apache/zeppelin/pull/4293#discussion_r795164516
##########
File path: zeppelin-plugins/launcher/yarn/src/main/java/org/apache/zeppelin/interpreter/launcher/YarnRemoteInterpreterProcess.java
##########
@@ -178,6 +178,10 @@ public void start(String userName) throws IOException {
this.appId = appResponse.getApplicationId();
ApplicationSubmissionContext appContext = yarnApplication.getApplicationSubmissionContext();
appContext = createApplicationSubmissionContext(appContext);
+ String YARN_NODE_LABEL_EXPRESSION = "zeppelin.interpreter.yarn.node.label.expression";
+ if(properties.containsKey(YARN_NODE_LABEL_EXPRESSION) && StringUtils.isNotEmpty(properties.getProperty(YARN_NODE_LABEL_EXPRESSION))) {
Review comment:
> You can use `properties.getProperty(key, defaultValue)` to simplify it
Yes, it's a good idea
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] huage1994 commented on a change in pull request #4293: [ZEPPELIN-5639] Support node label in yarn interpreter launcher
Posted by GitBox <gi...@apache.org>.
huage1994 commented on a change in pull request #4293:
URL: https://github.com/apache/zeppelin/pull/4293#discussion_r794992468
##########
File path: zeppelin-plugins/launcher/yarn/src/main/java/org/apache/zeppelin/interpreter/launcher/YarnRemoteInterpreterProcess.java
##########
@@ -178,6 +178,10 @@ public void start(String userName) throws IOException {
this.appId = appResponse.getApplicationId();
ApplicationSubmissionContext appContext = yarnApplication.getApplicationSubmissionContext();
appContext = createApplicationSubmissionContext(appContext);
+ String YARN_NODE_LABEL_EXPRESSION = "yarn.node.label.expression";
Review comment:
Thanks @zjffdu for the reviews. It's done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org