You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@twill.apache.org by chtyim <gi...@git.apache.org> on 2017/03/20 19:10:32 UTC
[GitHub] twill pull request #39: (TWILL-225) Refactor TwillPreprer to allow configura...
GitHub user chtyim opened a pull request:
https://github.com/apache/twill/pull/39
(TWILL-225) Refactor TwillPreprer to allow configurations affecting a\u2026
\u2026pplication behavior overridable per TwillPreprer
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/chtyim/twill feature/TWILL-225
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/twill/pull/39.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #39
----
commit babc2e1b6598319956b4333d3c3ef2ca3400a13c
Author: Terence Yim <ch...@apache.org>
Date: 2017-03-19T01:31:57Z
(TWILL-225) Refactor TwillPreprer to allow configurations affecting application behavior overridable per TwillPreprer
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] twill issue #39: (TWILL-225) Refactor TwillPreprer to allow configurations a...
Posted by anwar6953 <gi...@git.apache.org>.
Github user anwar6953 commented on the issue:
https://github.com/apache/twill/pull/39
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] twill pull request #39: (TWILL-225) Refactor TwillPreprer to allow configura...
Posted by chtyim <gi...@git.apache.org>.
Github user chtyim commented on a diff in the pull request:
https://github.com/apache/twill/pull/39#discussion_r107264780
--- Diff: twill-yarn/src/main/java/org/apache/twill/yarn/YarnTwillPreparer.java ---
@@ -455,21 +475,40 @@ private void saveLogLevels(String runnableName, Map<String, LogEntry.Level> logL
this.logLevels.put(runnableName, newLevels);
}
+ /**
+ * Creates an {@link Credentials} by copying the {@link Credentials} of the current user.
+ */
private Credentials createCredentials() {
Credentials credentials = new Credentials();
try {
credentials.addAll(UserGroupInformation.getCurrentUser().getCredentials());
+ } catch (IOException e) {
--- End diff --
I don't think so. Even if this call fail, if the caller is able to acquire required credentials in his own way and add them via the `addSecureStore` method, then the app can still be submitted and executed correctly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] twill pull request #39: (TWILL-225) Refactor TwillPreprer to allow configura...
Posted by yufeldman <gi...@git.apache.org>.
Github user yufeldman commented on a diff in the pull request:
https://github.com/apache/twill/pull/39#discussion_r107004427
--- Diff: twill-yarn/src/main/java/org/apache/twill/yarn/YarnTwillPreparer.java ---
@@ -155,25 +153,16 @@ public String apply(Class<?> cls) {
private ClassAcceptor classAcceptor;
private final Map<String, Integer> maxRetries = Maps.newHashMap();
- YarnTwillPreparer(YarnConfiguration yarnConfig, TwillSpecification twillSpec, RunId runId,
- YarnAppClient yarnAppClient, String zkConnectString, Location appLocation, Set<URL> twillClassPaths,
+ YarnTwillPreparer(Configuration config, TwillSpecification twillSpec, RunId runId,
--- End diff --
Any particular reason to switch to Configuration from YarnConfiguration?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] twill pull request #39: (TWILL-225) Refactor TwillPreprer to allow configura...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/twill/pull/39
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] twill pull request #39: (TWILL-225) Refactor TwillPreprer to allow configura...
Posted by anwar6953 <gi...@git.apache.org>.
Github user anwar6953 commented on a diff in the pull request:
https://github.com/apache/twill/pull/39#discussion_r107265236
--- Diff: twill-yarn/src/main/java/org/apache/twill/yarn/YarnTwillPreparer.java ---
@@ -455,21 +475,40 @@ private void saveLogLevels(String runnableName, Map<String, LogEntry.Level> logL
this.logLevels.put(runnableName, newLevels);
}
+ /**
+ * Creates an {@link Credentials} by copying the {@link Credentials} of the current user.
+ */
private Credentials createCredentials() {
Credentials credentials = new Credentials();
try {
credentials.addAll(UserGroupInformation.getCurrentUser().getCredentials());
+ } catch (IOException e) {
--- End diff --
I guess app submission would fail if caller doesn't acquire required credentials in its own way anyways.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] twill pull request #39: (TWILL-225) Refactor TwillPreprer to allow configura...
Posted by anwar6953 <gi...@git.apache.org>.
Github user anwar6953 commented on a diff in the pull request:
https://github.com/apache/twill/pull/39#discussion_r107262119
--- Diff: twill-yarn/src/main/java/org/apache/twill/yarn/YarnTwillPreparer.java ---
@@ -455,21 +475,40 @@ private void saveLogLevels(String runnableName, Map<String, LogEntry.Level> logL
this.logLevels.put(runnableName, newLevels);
}
+ /**
+ * Creates an {@link Credentials} by copying the {@link Credentials} of the current user.
+ */
private Credentials createCredentials() {
Credentials credentials = new Credentials();
try {
credentials.addAll(UserGroupInformation.getCurrentUser().getCredentials());
+ } catch (IOException e) {
--- End diff --
shouldn't this be fatal?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] twill pull request #39: (TWILL-225) Refactor TwillPreprer to allow configura...
Posted by chtyim <gi...@git.apache.org>.
Github user chtyim commented on a diff in the pull request:
https://github.com/apache/twill/pull/39#discussion_r107007777
--- Diff: twill-yarn/src/main/java/org/apache/twill/yarn/YarnTwillPreparer.java ---
@@ -155,25 +153,16 @@ public String apply(Class<?> cls) {
private ClassAcceptor classAcceptor;
private final Map<String, Integer> maxRetries = Maps.newHashMap();
- YarnTwillPreparer(YarnConfiguration yarnConfig, TwillSpecification twillSpec, RunId runId,
- YarnAppClient yarnAppClient, String zkConnectString, Location appLocation, Set<URL> twillClassPaths,
+ YarnTwillPreparer(Configuration config, TwillSpecification twillSpec, RunId runId,
--- End diff --
It's more generic. We actually not using any specific method from `YarnConfiguration` in this class.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---