You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by GitBox <gi...@apache.org> on 2020/09/27 07:49:15 UTC
[GitHub] [zeppelin] zjffdu opened a new pull request #3927: [ZEPPELIN-4409]. Set spark.app.name to know which user is running which notebook
zjffdu opened a new pull request #3927:
URL: https://github.com/apache/zeppelin/pull/3927
### What is this PR for?
Use the interpreter group id as the spark app name if it is not set. So that user can easily figure out which yarn app is what he is looking for no matter it is note isolated or user isolated.
### What type of PR is it?
[Improvement]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-4409
### How should this be tested?
* CI pass
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] asfgit closed pull request #3927: [ZEPPELIN-4409]. Set spark.app.name to know which user is running which notebook
Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #3927:
URL: https://github.com/apache/zeppelin/pull/3927
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] Reamer commented on a change in pull request #3927: [ZEPPELIN-4409]. Set spark.app.name to know which user is running which notebook
Posted by GitBox <gi...@apache.org>.
Reamer commented on a change in pull request #3927:
URL: https://github.com/apache/zeppelin/pull/3927#discussion_r502181777
##########
File path: zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/launcher/SparkInterpreterLauncher.java
##########
@@ -78,6 +78,15 @@ public SparkInterpreterLauncher(ZeppelinConfiguration zConf, RecoveryStorage rec
}
}
+ // set spark.app.name if it is not set or empty
+ if (!sparkProperties.containsKey("spark.app.name") ||
+ StringUtils.isBlank(sparkProperties.getProperty("spark.app.name"))) {
+ sparkProperties.setProperty("spark.app.name", context.getInterpreterGroupId());
+ } else {
+ String appName = context.getInterpreterGroupId();
+ sparkProperties.setProperty("spark.app.name", appName);
Review comment:
@zjffdu Why do you overwrite the value?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] zjffdu commented on a change in pull request #3927: [ZEPPELIN-4409]. Set spark.app.name to know which user is running which notebook
Posted by GitBox <gi...@apache.org>.
zjffdu commented on a change in pull request #3927:
URL: https://github.com/apache/zeppelin/pull/3927#discussion_r502244505
##########
File path: zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/launcher/SparkInterpreterLauncher.java
##########
@@ -78,6 +78,15 @@ public SparkInterpreterLauncher(ZeppelinConfiguration zConf, RecoveryStorage rec
}
}
+ // set spark.app.name if it is not set or empty
+ if (!sparkProperties.containsKey("spark.app.name") ||
+ StringUtils.isBlank(sparkProperties.getProperty("spark.app.name"))) {
+ sparkProperties.setProperty("spark.app.name", context.getInterpreterGroupId());
+ } else {
+ String appName = context.getInterpreterGroupId();
+ sparkProperties.setProperty("spark.app.name", appName);
Review comment:
oops, I should remove this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] zjffdu commented on pull request #3927: [ZEPPELIN-4409]. Set spark.app.name to know which user is running which notebook
Posted by GitBox <gi...@apache.org>.
zjffdu commented on pull request #3927:
URL: https://github.com/apache/zeppelin/pull/3927#issuecomment-705955503
Will merge if no more comment
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] zjffdu commented on pull request #3927: [ZEPPELIN-4409]. Set spark.app.name to know which user is running which notebook
Posted by GitBox <gi...@apache.org>.
zjffdu commented on pull request #3927:
URL: https://github.com/apache/zeppelin/pull/3927#issuecomment-705955503
Will merge if no more comment
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] Reamer commented on a change in pull request #3927: [ZEPPELIN-4409]. Set spark.app.name to know which user is running which notebook
Posted by GitBox <gi...@apache.org>.
Reamer commented on a change in pull request #3927:
URL: https://github.com/apache/zeppelin/pull/3927#discussion_r502181777
##########
File path: zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/launcher/SparkInterpreterLauncher.java
##########
@@ -78,6 +78,15 @@ public SparkInterpreterLauncher(ZeppelinConfiguration zConf, RecoveryStorage rec
}
}
+ // set spark.app.name if it is not set or empty
+ if (!sparkProperties.containsKey("spark.app.name") ||
+ StringUtils.isBlank(sparkProperties.getProperty("spark.app.name"))) {
+ sparkProperties.setProperty("spark.app.name", context.getInterpreterGroupId());
+ } else {
+ String appName = context.getInterpreterGroupId();
+ sparkProperties.setProperty("spark.app.name", appName);
Review comment:
@zjffdu Why do you overwrite the value?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zeppelin] zjffdu commented on a change in pull request #3927: [ZEPPELIN-4409]. Set spark.app.name to know which user is running which notebook
Posted by GitBox <gi...@apache.org>.
zjffdu commented on a change in pull request #3927:
URL: https://github.com/apache/zeppelin/pull/3927#discussion_r502244505
##########
File path: zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/launcher/SparkInterpreterLauncher.java
##########
@@ -78,6 +78,15 @@ public SparkInterpreterLauncher(ZeppelinConfiguration zConf, RecoveryStorage rec
}
}
+ // set spark.app.name if it is not set or empty
+ if (!sparkProperties.containsKey("spark.app.name") ||
+ StringUtils.isBlank(sparkProperties.getProperty("spark.app.name"))) {
+ sparkProperties.setProperty("spark.app.name", context.getInterpreterGroupId());
+ } else {
+ String appName = context.getInterpreterGroupId();
+ sparkProperties.setProperty("spark.app.name", appName);
Review comment:
oops, I should remove this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org