You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2020/01/17 16:31:07 UTC
[GitHub] [flink] walterddr opened a new pull request #10891:
[FLINK-15561][Security][hotfix] Add Delegation Token checker in
YarnClusterDescriptor
walterddr opened a new pull request #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
URL: https://github.com/apache/flink/pull/10891
…ker.
<!--
*Thank you very much for contributing to Apache Flink - we are happy that you want to help us improve Flink. To help the community review your contribution in the best possible way, please go through the checklist below, which will get the contribution into a shape in which it can be best reviewed.*
*Please understand that we do not do this to make contributions to Flink a hassle. In order to uphold a high standard of quality for code contributions, while at the same time managing a large number of contributions, we need contributors to prepare the contributions well, and give reviewers enough contextual information for the review. Please also understand that contributions that do not follow this guide will take longer to review and thus typically be picked up with lower priority by the community.*
## Contribution Checklist
- Make sure that the pull request corresponds to a [JIRA issue](https://issues.apache.org/jira/projects/FLINK/issues). Exceptions are made for typos in JavaDoc or documentation files, which need no JIRA issue.
## What is the purpose of the change
* YarnClusterDescriptor doesn't have the delegation token checker in security `HadoopModule`. Thus causes delegation token launch path problematically fails.
## Brief change log
add identical delegation token checker in YarnClusterDescriptor.
## Verifying this change
This change is already covered by existing tests.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: YARN
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? no
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot commented on issue #10891:
[FLINK-15561][Security][hotfix] Add Delegation Token checker in
YarnClusterDescriptor
Posted by GitBox <gi...@apache.org>.
flinkbot commented on issue #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
URL: https://github.com/apache/flink/pull/10891#issuecomment-575699403
Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
to review your pull request. We will use this comment to track the progress of the review.
## Automated Checks
Last check on commit 6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b (Fri Jan 17 16:34:00 UTC 2020)
**Warnings:**
* No documentation files were touched! Remember to keep the Flink docs up to date!
<sub>Mention the bot in a comment to re-run the automated checks.</sub>
## Review Progress
* ❓ 1. The [description] looks good.
* ❓ 2. There is [consensus] that the contribution should go into to Flink.
* ❓ 3. Needs [attention] from.
* ❓ 4. The change fits into the overall [architecture].
* ❓ 5. Overall code [quality] is good.
Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process.<details>
The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required <summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`)
- `@flinkbot approve all` to approve all aspects
- `@flinkbot approve-until architecture` to approve everything until `architecture`
- `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention
- `@flinkbot disapprove architecture` to remove an approval you gave earlier
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10891:
[FLINK-15561][Security][hotfix] Add Delegation Token checker in
YarnClusterDescriptor
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
URL: https://github.com/apache/flink/pull/10891#issuecomment-575711809
<!--
Meta data
Hash:6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/144974163 TriggerType:PUSH TriggerID:6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b
Hash:6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b Status:FAILURE URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4449 TriggerType:PUSH TriggerID:6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b
Hash:dad63dee9ca90a52192be11fa4d7024e5a3b584d Status:FAILURE URL:https://travis-ci.com/flink-ci/flink/builds/147775978 TriggerType:PUSH TriggerID:dad63dee9ca90a52192be11fa4d7024e5a3b584d
Hash:dad63dee9ca90a52192be11fa4d7024e5a3b584d Status:FAILURE URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4926 TriggerType:PUSH TriggerID:dad63dee9ca90a52192be11fa4d7024e5a3b584d
Hash:0dbe67fd3f66597bc50f3c0a0da7f2dd63287bee Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/148246420 TriggerType:PUSH TriggerID:0dbe67fd3f66597bc50f3c0a0da7f2dd63287bee
Hash:0dbe67fd3f66597bc50f3c0a0da7f2dd63287bee Status:FAILURE URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5020 TriggerType:PUSH TriggerID:0dbe67fd3f66597bc50f3c0a0da7f2dd63287bee
-->
## CI report:
* 6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144974163) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4449)
* dad63dee9ca90a52192be11fa4d7024e5a3b584d Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/147775978) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4926)
* 0dbe67fd3f66597bc50f3c0a0da7f2dd63287bee Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/148246420) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5020)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10891:
[FLINK-15561][Security][hotfix] Add Delegation Token checker in
YarnClusterDescriptor
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
URL: https://github.com/apache/flink/pull/10891#issuecomment-575711809
<!--
Meta data
Hash:6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/144974163 TriggerType:PUSH TriggerID:6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b
Hash:6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b Status:FAILURE URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4449 TriggerType:PUSH TriggerID:6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b
-->
## CI report:
* 6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144974163) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4449)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] walterddr commented on issue #10891:
[FLINK-15561][Security][hotfix] Add Delegation Token checker in
YarnClusterDescriptor
Posted by GitBox <gi...@apache.org>.
walterddr commented on issue #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
URL: https://github.com/apache/flink/pull/10891#issuecomment-580836561
@tillrohrmann could you please kindly take a look and see if this fix makes sense?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10891:
[FLINK-15561][Security][hotfix] Add Delegation Token checker in
YarnClusterDescriptor
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
URL: https://github.com/apache/flink/pull/10891#issuecomment-575711809
<!--
Meta data
Hash:6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/144974163 TriggerType:PUSH TriggerID:6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b
Hash:6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b Status:FAILURE URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4449 TriggerType:PUSH TriggerID:6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b
Hash:dad63dee9ca90a52192be11fa4d7024e5a3b584d Status:FAILURE URL:https://travis-ci.com/flink-ci/flink/builds/147775978 TriggerType:PUSH TriggerID:dad63dee9ca90a52192be11fa4d7024e5a3b584d
Hash:dad63dee9ca90a52192be11fa4d7024e5a3b584d Status:FAILURE URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4926 TriggerType:PUSH TriggerID:dad63dee9ca90a52192be11fa4d7024e5a3b584d
-->
## CI report:
* 6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144974163) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4449)
* dad63dee9ca90a52192be11fa4d7024e5a3b584d Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/147775978) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4926)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] walterddr commented on a change in pull request #10891:
[FLINK-15561][Security][hotfix] Add Delegation Token checker in
YarnClusterDescriptor
Posted by GitBox <gi...@apache.org>.
walterddr commented on a change in pull request #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
URL: https://github.com/apache/flink/pull/10891#discussion_r377198538
##########
File path: flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/util/HadoopUtils.java
##########
@@ -112,6 +112,27 @@ public static Configuration getHadoopConfiguration(org.apache.flink.configuratio
return result;
}
+ public static boolean isCredentialsConfigured(boolean useTicketCache) throws Exception {
+ UserGroupInformation loginUser = UserGroupInformation.getCurrentUser();
Review comment:
It's a bit tricky after a second look. sometimes Flink expects the actual "logged in" user with valid keytab/web-auth so that it can generate additional DT. I am thinking of opening another JIRA to address it as a separate issue; meanwhile I will keep this PR solely fixing the DT error check. what do you think?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] walterddr commented on issue #10891:
[FLINK-15561][Security][hotfix] Add Delegation Token checker in
YarnClusterDescriptor
Posted by GitBox <gi...@apache.org>.
walterddr commented on issue #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
URL: https://github.com/apache/flink/pull/10891#issuecomment-582506119
> This should fix the original issue outlined in https://lists.apache.org/thread.html/49c0d8c011daa227287473f6600f6f86dbb11a0afe8a1bdce107a0ce%40%3Cuser.flink.apache.org%3E, right?
I couldn't reproduce this exception in our security environment actually, but I suspect this could be the issue. I would reply to the ML thread and ask owner for a verification.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] aljoscha commented on a change in pull request #10891:
[FLINK-15561][Security][hotfix] Add Delegation Token checker in
YarnClusterDescriptor
Posted by GitBox <gi...@apache.org>.
aljoscha commented on a change in pull request #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
URL: https://github.com/apache/flink/pull/10891#discussion_r374031205
##########
File path: flink-yarn/src/main/java/org/apache/flink/yarn/YarnClusterDescriptor.java
##########
@@ -435,9 +436,13 @@ public void killCluster(ApplicationId applicationId) throws FlinkException {
UserGroupInformation loginUser = UserGroupInformation.getCurrentUser();
if (loginUser.getAuthenticationMethod() == UserGroupInformation.AuthenticationMethod.KERBEROS
&& useTicketCache && !loginUser.hasKerberosCredentials()) {
- LOG.error("Hadoop security with Kerberos is enabled but the login user does not have Kerberos credentials");
- throw new RuntimeException("Hadoop security with Kerberos is enabled but the login user " +
+ // a delegation token is an adequate substitute in most cases
Review comment:
We could also move the whole block, i.e.
```
if (securityConfig.useTicketCache() && !loginUser.hasKerberosCredentials()) {
// a delegation token is an adequate substitute in most cases
if (!HadoopUtils.hasHDFSDelegationToken()) {
LOG.warn("Hadoop security is enabled but current login user does not have Kerberos credentials");
}
}
```
to `HadoopUtils` and use it in both `HadoopModule` and this code, thus reducing code duplication.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] aljoscha closed pull request #10891:
[FLINK-15561][Security][hotfix] Add Delegation Token checker in
YarnClusterDescriptor
Posted by GitBox <gi...@apache.org>.
aljoscha closed pull request #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
URL: https://github.com/apache/flink/pull/10891
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] aljoscha commented on a change in pull request #10891:
[FLINK-15561][Security][hotfix] Add Delegation Token checker in
YarnClusterDescriptor
Posted by GitBox <gi...@apache.org>.
aljoscha commented on a change in pull request #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
URL: https://github.com/apache/flink/pull/10891#discussion_r377155239
##########
File path: flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/util/HadoopUtils.java
##########
@@ -112,6 +112,27 @@ public static Configuration getHadoopConfiguration(org.apache.flink.configuratio
return result;
}
+ public static boolean isCredentialsConfigured(boolean useTicketCache) throws Exception {
+ UserGroupInformation loginUser = UserGroupInformation.getCurrentUser();
Review comment:
So it seems we should use `getCurrentUser` here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] walterddr commented on a change in pull request #10891:
[FLINK-15561][Security][hotfix] Add Delegation Token checker in
YarnClusterDescriptor
Posted by GitBox <gi...@apache.org>.
walterddr commented on a change in pull request #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
URL: https://github.com/apache/flink/pull/10891#discussion_r368648299
##########
File path: flink-yarn/src/main/java/org/apache/flink/yarn/YarnClusterDescriptor.java
##########
@@ -435,9 +436,13 @@ public void killCluster(ApplicationId applicationId) throws FlinkException {
UserGroupInformation loginUser = UserGroupInformation.getCurrentUser();
if (loginUser.getAuthenticationMethod() == UserGroupInformation.AuthenticationMethod.KERBEROS
&& useTicketCache && !loginUser.hasKerberosCredentials()) {
- LOG.error("Hadoop security with Kerberos is enabled but the login user does not have Kerberos credentials");
- throw new RuntimeException("Hadoop security with Kerberos is enabled but the login user " +
+ // a delegation token is an adequate substitute in most cases
Review comment:
Yes. this is the expected result. See: https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/security/modules/HadoopModule.java#L146 which is used in client/JM/TM.
However I think the checker in here not present - this means specifically in YarnClusterDescriptor it does not support delegation token. This seems like a bug to me.
Regarding the implementation, yes I can do that as well (e.g. merging with the upper layer if, I was trying to pair the code logic with `HadoopModule`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] walterddr commented on a change in pull request #10891:
[FLINK-15561][Security][hotfix] Add Delegation Token checker in
YarnClusterDescriptor
Posted by GitBox <gi...@apache.org>.
walterddr commented on a change in pull request #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
URL: https://github.com/apache/flink/pull/10891#discussion_r375381727
##########
File path: flink-yarn/src/main/java/org/apache/flink/yarn/YarnClusterDescriptor.java
##########
@@ -435,9 +436,13 @@ public void killCluster(ApplicationId applicationId) throws FlinkException {
UserGroupInformation loginUser = UserGroupInformation.getCurrentUser();
if (loginUser.getAuthenticationMethod() == UserGroupInformation.AuthenticationMethod.KERBEROS
&& useTicketCache && !loginUser.hasKerberosCredentials()) {
- LOG.error("Hadoop security with Kerberos is enabled but the login user does not have Kerberos credentials");
- throw new RuntimeException("Hadoop security with Kerberos is enabled but the login user " +
+ // a delegation token is an adequate substitute in most cases
Review comment:
+1 moving to HadoopUtils makes more sense to me actually - if I understand correctly, delegation Token is a hadoop concept; Kerberos official doc seems to only contain a guidance on credential delegation: https://tools.ietf.org/html/rfc5896.html.
I will make the adjustment accordingly
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] walterddr commented on a change in pull request #10891:
[FLINK-15561][Security][hotfix] Add Delegation Token checker in
YarnClusterDescriptor
Posted by GitBox <gi...@apache.org>.
walterddr commented on a change in pull request #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
URL: https://github.com/apache/flink/pull/10891#discussion_r376642304
##########
File path: flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/util/HadoopUtils.java
##########
@@ -112,6 +112,27 @@ public static Configuration getHadoopConfiguration(org.apache.flink.configuratio
return result;
}
+ public static boolean isCredentialsConfigured(boolean useTicketCache) throws Exception {
+ UserGroupInformation loginUser = UserGroupInformation.getCurrentUser();
Review comment:
Hmm. I might be zeroing in on this problem. Although the [documentation](https://hadoop.apache.org/docs/r2.8.3/api/org/apache/hadoop/security/UserGroupInformation.html) is pretty vague --> seems like `getLoginUser` returns the actual user logged in via keytab or ticket; and `getCurrentUser` return the actual user (login, or the impersonated user via delegation token).
In short, say I am logged in as userA and then use a `UGI.doAs` section as a proxy userB. inside the `doAs` section. getLoginUser returns userA and getCurrentUser returns userB.. This might be the cause of the delegation token based security problem
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10891:
[FLINK-15561][Security][hotfix] Add Delegation Token checker in
YarnClusterDescriptor
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
URL: https://github.com/apache/flink/pull/10891#issuecomment-575711809
<!--
Meta data
Hash:6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/144974163 TriggerType:PUSH TriggerID:6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b
Hash:6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b Status:FAILURE URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4449 TriggerType:PUSH TriggerID:6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b
Hash:dad63dee9ca90a52192be11fa4d7024e5a3b584d Status:FAILURE URL:https://travis-ci.com/flink-ci/flink/builds/147775978 TriggerType:PUSH TriggerID:dad63dee9ca90a52192be11fa4d7024e5a3b584d
Hash:dad63dee9ca90a52192be11fa4d7024e5a3b584d Status:FAILURE URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4926 TriggerType:PUSH TriggerID:dad63dee9ca90a52192be11fa4d7024e5a3b584d
Hash:0dbe67fd3f66597bc50f3c0a0da7f2dd63287bee Status:UNKNOWN URL:TBD TriggerType:PUSH TriggerID:0dbe67fd3f66597bc50f3c0a0da7f2dd63287bee
-->
## CI report:
* 6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144974163) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4449)
* dad63dee9ca90a52192be11fa4d7024e5a3b584d Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/147775978) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4926)
* 0dbe67fd3f66597bc50f3c0a0da7f2dd63287bee UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] aljoscha commented on a change in pull request #10891:
[FLINK-15561][Security][hotfix] Add Delegation Token checker in
YarnClusterDescriptor
Posted by GitBox <gi...@apache.org>.
aljoscha commented on a change in pull request #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
URL: https://github.com/apache/flink/pull/10891#discussion_r376279727
##########
File path: flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/util/HadoopUtils.java
##########
@@ -112,6 +112,27 @@ public static Configuration getHadoopConfiguration(org.apache.flink.configuratio
return result;
}
+ public static boolean isCredentialsConfigured(boolean useTicketCache) throws Exception {
+ UserGroupInformation loginUser = UserGroupInformation.getCurrentUser();
Review comment:
The previous code used `getLoginUser()` (in `HadoopModule`) and `getCurrentUser()` (in `YarnClusterDescriptor), is it ok to only use `getCurrentUser()` now or should we maybe add the UGI as a parameter?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] aljoscha commented on a change in pull request #10891:
[FLINK-15561][Security][hotfix] Add Delegation Token checker in
YarnClusterDescriptor
Posted by GitBox <gi...@apache.org>.
aljoscha commented on a change in pull request #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
URL: https://github.com/apache/flink/pull/10891#discussion_r376280166
##########
File path: flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/util/HadoopUtils.java
##########
@@ -112,6 +112,27 @@ public static Configuration getHadoopConfiguration(org.apache.flink.configuratio
return result;
}
+ public static boolean isCredentialsConfigured(boolean useTicketCache) throws Exception {
+ UserGroupInformation loginUser = UserGroupInformation.getCurrentUser();
+ if (UserGroupInformation.isSecurityEnabled()) {
+ // note: UGI::hasKerberosCredentials inaccurately reports false
+ // for logins based on a keytab (fixed in Hadoop 2.6.1, see HADOOP-10786),
+ // so we check only in ticket cache scenario.
+ if (useTicketCache && !loginUser.hasKerberosCredentials()) {
+ // a delegation token is an adequate substitute in most cases
+ if (!HadoopUtils.hasHDFSDelegationToken()) {
+ LOG.warn("Hadoop security is enabled but current login user does not have Kerberos credentials, " +
Review comment:
Are the two branches in the wrong order here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10891:
[FLINK-15561][Security][hotfix] Add Delegation Token checker in
YarnClusterDescriptor
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
URL: https://github.com/apache/flink/pull/10891#issuecomment-575711809
<!--
Meta data
Hash:6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/144974163 TriggerType:PUSH TriggerID:6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b
Hash:6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b Status:FAILURE URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4449 TriggerType:PUSH TriggerID:6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b
Hash:dad63dee9ca90a52192be11fa4d7024e5a3b584d Status:UNKNOWN URL:TBD TriggerType:PUSH TriggerID:dad63dee9ca90a52192be11fa4d7024e5a3b584d
-->
## CI report:
* 6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144974163) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4449)
* dad63dee9ca90a52192be11fa4d7024e5a3b584d UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10891:
[FLINK-15561][Security][hotfix] Add Delegation Token checker in
YarnClusterDescriptor
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
URL: https://github.com/apache/flink/pull/10891#issuecomment-575711809
<!--
Meta data
Hash:6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b Status:PENDING URL:https://travis-ci.com/flink-ci/flink/builds/144974163 TriggerType:PUSH TriggerID:6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b
Hash:6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b Status:PENDING URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4449 TriggerType:PUSH TriggerID:6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b
-->
## CI report:
* 6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/144974163) Azure: [PENDING](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4449)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] aljoscha commented on issue #10891:
[FLINK-15561][Security][hotfix] Add Delegation Token checker in
YarnClusterDescriptor
Posted by GitBox <gi...@apache.org>.
aljoscha commented on issue #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
URL: https://github.com/apache/flink/pull/10891#issuecomment-581346296
This should fix the original issue outlined in https://lists.apache.org/thread.html/49c0d8c011daa227287473f6600f6f86dbb11a0afe8a1bdce107a0ce%40%3Cuser.flink.apache.org%3E, right?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] walterddr commented on a change in pull request #10891:
[FLINK-15561][Security][hotfix] Add Delegation Token checker in
YarnClusterDescriptor
Posted by GitBox <gi...@apache.org>.
walterddr commented on a change in pull request #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
URL: https://github.com/apache/flink/pull/10891#discussion_r376642304
##########
File path: flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/util/HadoopUtils.java
##########
@@ -112,6 +112,27 @@ public static Configuration getHadoopConfiguration(org.apache.flink.configuratio
return result;
}
+ public static boolean isCredentialsConfigured(boolean useTicketCache) throws Exception {
+ UserGroupInformation loginUser = UserGroupInformation.getCurrentUser();
Review comment:
Hmm. I might be zeroing in on this problem. Although the [documentation](https://hadoop.apache.org/docs/r2.8.3/api/org/apache/hadoop/security/UserGroupInformation.html) is pretty vague --> seems like `getLoginUser` returns the actual user logged in via keytab or ticket; and `getCurrentUser` return the actual user (login, or the impersonated user via delegation token).
In short, say I am logged in as userA and then use a `UGI.doAs` section as a proxy userB. inside the `doAs` section. getLoginUser returns userA and getCurrentUser returns userB.. This might be the cause of the delegation token based security problem -- it might result in UGI differences
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] walterddr edited a comment on issue #10891:
[FLINK-15561][Security][hotfix] Add Delegation Token checker in
YarnClusterDescriptor
Posted by GitBox <gi...@apache.org>.
walterddr edited a comment on issue #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
URL: https://github.com/apache/flink/pull/10891#issuecomment-582506119
> This should fix the original issue outlined in https://lists.apache.org/thread.html/49c0d8c011daa227287473f6600f6f86dbb11a0afe8a1bdce107a0ce%40%3Cuser.flink.apache.org%3E, right?
I couldn't reproduce this exception in our security environment actually, but I suspect this could be the issue. I would reply to the ML thread and ask the question owner and see if he can provide a verification.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10891:
[FLINK-15561][Security][hotfix] Add Delegation Token checker in
YarnClusterDescriptor
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
URL: https://github.com/apache/flink/pull/10891#issuecomment-575711809
<!--
Meta data
Hash:6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b Status:PENDING URL:https://travis-ci.com/flink-ci/flink/builds/144974163 TriggerType:PUSH TriggerID:6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b
Hash:6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b Status:FAILURE URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4449 TriggerType:PUSH TriggerID:6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b
-->
## CI report:
* 6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/144974163) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4449)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot commented on issue #10891:
[FLINK-15561][Security][hotfix] Add Delegation Token checker in
YarnClusterDescriptor
Posted by GitBox <gi...@apache.org>.
flinkbot commented on issue #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
URL: https://github.com/apache/flink/pull/10891#issuecomment-575711809
<!--
Meta data
Hash:6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b Status:UNKNOWN URL:TBD TriggerType:PUSH TriggerID:6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b
-->
## CI report:
* 6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #10891:
[FLINK-15561][Security][hotfix] Add Delegation Token checker in
YarnClusterDescriptor
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on issue #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
URL: https://github.com/apache/flink/pull/10891#issuecomment-575711809
<!--
Meta data
Hash:6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b Status:SUCCESS URL:https://travis-ci.com/flink-ci/flink/builds/144974163 TriggerType:PUSH TriggerID:6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b
Hash:6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b Status:FAILURE URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4449 TriggerType:PUSH TriggerID:6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b
Hash:dad63dee9ca90a52192be11fa4d7024e5a3b584d Status:FAILURE URL:https://travis-ci.com/flink-ci/flink/builds/147775978 TriggerType:PUSH TriggerID:dad63dee9ca90a52192be11fa4d7024e5a3b584d
Hash:dad63dee9ca90a52192be11fa4d7024e5a3b584d Status:FAILURE URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4926 TriggerType:PUSH TriggerID:dad63dee9ca90a52192be11fa4d7024e5a3b584d
Hash:0dbe67fd3f66597bc50f3c0a0da7f2dd63287bee Status:PENDING URL:https://travis-ci.com/flink-ci/flink/builds/148246420 TriggerType:PUSH TriggerID:0dbe67fd3f66597bc50f3c0a0da7f2dd63287bee
Hash:0dbe67fd3f66597bc50f3c0a0da7f2dd63287bee Status:FAILURE URL:https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5020 TriggerType:PUSH TriggerID:0dbe67fd3f66597bc50f3c0a0da7f2dd63287bee
-->
## CI report:
* 6de7e8f4a6b893f0c4e731efa68ee78d144e6a4b Travis: [SUCCESS](https://travis-ci.com/flink-ci/flink/builds/144974163) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4449)
* dad63dee9ca90a52192be11fa4d7024e5a3b584d Travis: [FAILURE](https://travis-ci.com/flink-ci/flink/builds/147775978) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=4926)
* 0dbe67fd3f66597bc50f3c0a0da7f2dd63287bee Travis: [PENDING](https://travis-ci.com/flink-ci/flink/builds/148246420) Azure: [FAILURE](https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_build/results?buildId=5020)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] aljoscha commented on issue #10891:
[FLINK-15561][Security][hotfix] Add Delegation Token checker in
YarnClusterDescriptor
Posted by GitBox <gi...@apache.org>.
aljoscha commented on issue #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
URL: https://github.com/apache/flink/pull/10891#issuecomment-581349701
I think the fix is good, we just have to agree on where the code should be. 😅We could think about adding a check for this to the end to end test, but that would be a bigger endeavour, I think.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] walterddr commented on a change in pull request #10891:
[FLINK-15561][Security][hotfix] Add Delegation Token checker in
YarnClusterDescriptor
Posted by GitBox <gi...@apache.org>.
walterddr commented on a change in pull request #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
URL: https://github.com/apache/flink/pull/10891#discussion_r377198538
##########
File path: flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/util/HadoopUtils.java
##########
@@ -112,6 +112,27 @@ public static Configuration getHadoopConfiguration(org.apache.flink.configuratio
return result;
}
+ public static boolean isCredentialsConfigured(boolean useTicketCache) throws Exception {
+ UserGroupInformation loginUser = UserGroupInformation.getCurrentUser();
Review comment:
It's a bit tricky after a second look. sometimes Flink actually expects the actual "logged in" user with valid keytab/web-auth so that it can generate additional DT. I am thinking of opening another JIRA to address it as a separate issue; meanwhile I will keep this PR solely fixing the DT error check. what do you think?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] aljoscha commented on a change in pull request #10891:
[FLINK-15561][Security][hotfix] Add Delegation Token checker in
YarnClusterDescriptor
Posted by GitBox <gi...@apache.org>.
aljoscha commented on a change in pull request #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
URL: https://github.com/apache/flink/pull/10891#discussion_r374031205
##########
File path: flink-yarn/src/main/java/org/apache/flink/yarn/YarnClusterDescriptor.java
##########
@@ -435,9 +436,13 @@ public void killCluster(ApplicationId applicationId) throws FlinkException {
UserGroupInformation loginUser = UserGroupInformation.getCurrentUser();
if (loginUser.getAuthenticationMethod() == UserGroupInformation.AuthenticationMethod.KERBEROS
&& useTicketCache && !loginUser.hasKerberosCredentials()) {
- LOG.error("Hadoop security with Kerberos is enabled but the login user does not have Kerberos credentials");
- throw new RuntimeException("Hadoop security with Kerberos is enabled but the login user " +
+ // a delegation token is an adequate substitute in most cases
Review comment:
We could also move the whole block, i.e (from `HadoopModule`).
```
if (securityConfig.useTicketCache() && !loginUser.hasKerberosCredentials()) {
// a delegation token is an adequate substitute in most cases
if (!HadoopUtils.hasHDFSDelegationToken()) {
LOG.warn("Hadoop security is enabled but current login user does not have Kerberos credentials");
}
}
```
to `HadoopUtils` and use it in both `HadoopModule` and this code, thus reducing code duplication.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] yanghua commented on a change in pull request #10891:
[FLINK-15561][Security][hotfix] Add Delegation Token checker in
YarnClusterDescriptor
Posted by GitBox <gi...@apache.org>.
yanghua commented on a change in pull request #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
URL: https://github.com/apache/flink/pull/10891#discussion_r368403224
##########
File path: flink-yarn/src/main/java/org/apache/flink/yarn/YarnClusterDescriptor.java
##########
@@ -435,9 +436,13 @@ public void killCluster(ApplicationId applicationId) throws FlinkException {
UserGroupInformation loginUser = UserGroupInformation.getCurrentUser();
if (loginUser.getAuthenticationMethod() == UserGroupInformation.AuthenticationMethod.KERBEROS
&& useTicketCache && !loginUser.hasKerberosCredentials()) {
- LOG.error("Hadoop security with Kerberos is enabled but the login user does not have Kerberos credentials");
- throw new RuntimeException("Hadoop security with Kerberos is enabled but the login user " +
+ // a delegation token is an adequate substitute in most cases
Review comment:
You added a stricter condition to the exception identification. Comparing with the origin logic, if it does not meet the conditions you added, the original exception will not be logged and thrown. Is this what you expected? In addition, if you just want to identify the exception more strictly, why not add it to the outer if block together?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [flink] aljoscha commented on issue #10891:
[FLINK-15561][Security][hotfix] Add Delegation Token checker in
YarnClusterDescriptor
Posted by GitBox <gi...@apache.org>.
aljoscha commented on issue #10891: [FLINK-15561][Security][hotfix] Add Delegation Token checker in YarnClusterDescriptor
URL: https://github.com/apache/flink/pull/10891#issuecomment-584546464
Thanks! I merged this. It's a good refactor, even though it doesn't yet solve the original user problem.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services