You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by GitBox <gi...@apache.org> on 2020/10/28 20:14:48 UTC
[GitHub] [nifi] tpalfy opened a new pull request #4629: NIFI-7954 Wrapping HBase_*_ClientService calls in getUgi().doAs()
tpalfy opened a new pull request #4629:
URL: https://github.com/apache/nifi/pull/4629
Taking care of TGT renewal.
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-XXXX** where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [ ] Has your PR been rebased against the latest commit within the target branch (typically `main`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not `squash` or use `--force` when pushing to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn -Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main `LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main `NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to .name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for build issues and submit an update to your PR as soon as possible.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] bbende commented on a change in pull request #4629: NIFI-7954 Wrapping HBase_*_ClientService calls in getUgi().doAs()
Posted by GitBox <gi...@apache.org>.
bbende commented on a change in pull request #4629:
URL: https://github.com/apache/nifi/pull/4629#discussion_r514338031
##########
File path: nifi-nar-bundles/nifi-extension-utils/nifi-hadoop-utils/src/main/java/org/apache/nifi/hadoop/SecurityUtil.java
##########
@@ -141,4 +143,37 @@ public static boolean isSecurityEnabled(final Configuration config) {
Validate.notNull(config);
return KERBEROS.equalsIgnoreCase(config.get(HADOOP_SECURITY_AUTHENTICATION));
}
+
+ public static <T> T callWithUgi(CallableThrowingException<UserGroupInformation, IOException> ugiProvider, CallableThrowingException<T, IOException> function) throws IOException {
Review comment:
I haven't looked too closely, but I was also wondering about the need for `CallableThrowingException `, isn't it basically providing the same thing as `PrivilegedExceptionAction` ?
I would think the code inside the client services could just be:
```
getUgi().doAs((PrivilegedExceptionAction<Object>)() -> {
...
}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] adenes commented on a change in pull request #4629: NIFI-7954 Wrapping HBase_*_ClientService calls in getUgi().doAs()
Posted by GitBox <gi...@apache.org>.
adenes commented on a change in pull request #4629:
URL: https://github.com/apache/nifi/pull/4629#discussion_r514300383
##########
File path: nifi-nar-bundles/nifi-extension-utils/nifi-hadoop-utils/src/main/java/org/apache/nifi/hadoop/SecurityUtil.java
##########
@@ -141,4 +143,37 @@ public static boolean isSecurityEnabled(final Configuration config) {
Validate.notNull(config);
return KERBEROS.equalsIgnoreCase(config.get(HADOOP_SECURITY_AUTHENTICATION));
}
+
+ public static <T> T callWithUgi(CallableThrowingException<UserGroupInformation, IOException> ugiProvider, CallableThrowingException<T, IOException> function) throws IOException {
Review comment:
In my opinion passing in a `UserGroupProvider` instance instead of the `ugiProvider` would make no difference from the functionality point of view but it'd simplify the usage/improve the readability.
As for the second parameter I'm not fully sure why we need the `CallableThrowingException` class. For me simple `Callable` seems to be ok too, even though it doesn't specify its exception type, but I don't see the need for that.
##########
File path: nifi-nar-bundles/nifi-extension-utils/nifi-hadoop-utils/src/main/java/org/apache/nifi/hadoop/SecurityUtil.java
##########
@@ -141,4 +143,37 @@ public static boolean isSecurityEnabled(final Configuration config) {
Validate.notNull(config);
return KERBEROS.equalsIgnoreCase(config.get(HADOOP_SECURITY_AUTHENTICATION));
}
+
+ public static <T> T callWithUgi(CallableThrowingException<UserGroupInformation, IOException> ugiProvider, CallableThrowingException<T, IOException> function) throws IOException {
+ try {
+ return ugiProvider.call().doAs((PrivilegedExceptionAction<T>)() -> function.call());
Review comment:
minor nit: `() -> function.call()` could be replaced with method reference, i.e. `function::call`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] tpalfy commented on a change in pull request #4629: NIFI-7954 Wrapping HBase_*_ClientService calls in getUgi().doAs()
Posted by GitBox <gi...@apache.org>.
tpalfy commented on a change in pull request #4629:
URL: https://github.com/apache/nifi/pull/4629#discussion_r518690436
##########
File path: nifi-nar-bundles/nifi-extension-utils/nifi-hadoop-utils/src/main/java/org/apache/nifi/hadoop/SecurityUtil.java
##########
@@ -141,4 +143,37 @@ public static boolean isSecurityEnabled(final Configuration config) {
Validate.notNull(config);
return KERBEROS.equalsIgnoreCase(config.get(HADOOP_SECURITY_AUTHENTICATION));
}
+
+ public static <T> T callWithUgi(UserGroupInformation ugi, PrivilegedExceptionAction<T> action) throws IOException {
+ try {
+ return ugi.doAs(action);
+ } catch (InterruptedException e) {
+ throw new IOException(e);
+ }
+ }
+
+ public static void checkTGTAndRelogin(ComponentLog log, KerberosUser kerberosUser, UserGroupInformation ugi) throws IOException {
Review comment:
Indeed, removing it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] adenes commented on a change in pull request #4629: NIFI-7954 Wrapping HBase_*_ClientService calls in getUgi().doAs()
Posted by GitBox <gi...@apache.org>.
adenes commented on a change in pull request #4629:
URL: https://github.com/apache/nifi/pull/4629#discussion_r514307394
##########
File path: nifi-nar-bundles/nifi-extension-utils/nifi-hadoop-utils/src/main/java/org/apache/nifi/hadoop/SecurityUtil.java
##########
@@ -141,4 +143,37 @@ public static boolean isSecurityEnabled(final Configuration config) {
Validate.notNull(config);
return KERBEROS.equalsIgnoreCase(config.get(HADOOP_SECURITY_AUTHENTICATION));
}
+
+ public static <T> T callWithUgi(CallableThrowingException<UserGroupInformation, IOException> ugiProvider, CallableThrowingException<T, IOException> function) throws IOException {
Review comment:
wow, the first point from this comment has been fixed even before I submitted the comment :) Thanks @tpalfy
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] tpalfy commented on a change in pull request #4629: NIFI-7954 Wrapping HBase_*_ClientService calls in getUgi().doAs()
Posted by GitBox <gi...@apache.org>.
tpalfy commented on a change in pull request #4629:
URL: https://github.com/apache/nifi/pull/4629#discussion_r518692323
##########
File path: nifi-nar-bundles/nifi-extension-utils/nifi-hadoop-utils/src/main/java/org/apache/nifi/hadoop/SecurityUtil.java
##########
@@ -141,4 +143,37 @@ public static boolean isSecurityEnabled(final Configuration config) {
Validate.notNull(config);
return KERBEROS.equalsIgnoreCase(config.get(HADOOP_SECURITY_AUTHENTICATION));
}
+
+ public static <T> T callWithUgi(UserGroupInformation ugi, PrivilegedExceptionAction<T> action) throws IOException {
Review comment:
It handles the `InterruptedException`. Most of the callers only declare `IOException`. This is a general pattern which could use a utility method in my opinion. Just in this case the wrapping would need to be replicated in 15 places or so.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] bbende merged pull request #4629: NIFI-7954 Wrapping HBase_*_ClientService calls in getUgi().doAs()
Posted by GitBox <gi...@apache.org>.
bbende merged pull request #4629:
URL: https://github.com/apache/nifi/pull/4629
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] bbende commented on a change in pull request #4629: NIFI-7954 Wrapping HBase_*_ClientService calls in getUgi().doAs()
Posted by GitBox <gi...@apache.org>.
bbende commented on a change in pull request #4629:
URL: https://github.com/apache/nifi/pull/4629#discussion_r519832920
##########
File path: nifi-nar-bundles/nifi-extension-utils/nifi-hadoop-utils/src/main/java/org/apache/nifi/hadoop/SecurityUtil.java
##########
@@ -141,4 +143,37 @@ public static boolean isSecurityEnabled(final Configuration config) {
Validate.notNull(config);
return KERBEROS.equalsIgnoreCase(config.get(HADOOP_SECURITY_AUTHENTICATION));
}
+
+ public static <T> T callWithUgi(UserGroupInformation ugi, PrivilegedExceptionAction<T> action) throws IOException {
Review comment:
Ok fair enough
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] tpalfy commented on a change in pull request #4629: NIFI-7954 Wrapping HBase_*_ClientService calls in getUgi().doAs()
Posted by GitBox <gi...@apache.org>.
tpalfy commented on a change in pull request #4629:
URL: https://github.com/apache/nifi/pull/4629#discussion_r515980827
##########
File path: nifi-nar-bundles/nifi-extension-utils/nifi-hadoop-utils/src/main/java/org/apache/nifi/hadoop/SecurityUtil.java
##########
@@ -141,4 +143,37 @@ public static boolean isSecurityEnabled(final Configuration config) {
Validate.notNull(config);
return KERBEROS.equalsIgnoreCase(config.get(HADOOP_SECURITY_AUTHENTICATION));
}
+
+ public static <T> T callWithUgi(CallableThrowingException<UserGroupInformation, IOException> ugiProvider, CallableThrowingException<T, IOException> function) throws IOException {
Review comment:
Yeah, that would work. I overcomplicated it a little it seems. Updated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] tpalfy commented on a change in pull request #4629: NIFI-7954 Wrapping HBase_*_ClientService calls in getUgi().doAs()
Posted by GitBox <gi...@apache.org>.
tpalfy commented on a change in pull request #4629:
URL: https://github.com/apache/nifi/pull/4629#discussion_r518690251
##########
File path: nifi-nar-bundles/nifi-extension-utils/nifi-hadoop-utils/src/main/java/org/apache/nifi/hadoop/CallableThrowingException.java
##########
@@ -0,0 +1,22 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.hadoop;
+
+@FunctionalInterface
+public interface CallableThrowingException<T, E extends Exception> {
Review comment:
Yes, deleting it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] bbende commented on pull request #4629: NIFI-7954 Wrapping HBase_*_ClientService calls in getUgi().doAs()
Posted by GitBox <gi...@apache.org>.
bbende commented on pull request #4629:
URL: https://github.com/apache/nifi/pull/4629#issuecomment-724030857
Looks good, going to merge
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] bbende commented on a change in pull request #4629: NIFI-7954 Wrapping HBase_*_ClientService calls in getUgi().doAs()
Posted by GitBox <gi...@apache.org>.
bbende commented on a change in pull request #4629:
URL: https://github.com/apache/nifi/pull/4629#discussion_r516688495
##########
File path: nifi-nar-bundles/nifi-extension-utils/nifi-hadoop-utils/src/main/java/org/apache/nifi/hadoop/CallableThrowingException.java
##########
@@ -0,0 +1,22 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.hadoop;
+
+@FunctionalInterface
+public interface CallableThrowingException<T, E extends Exception> {
Review comment:
This can be removed now that it is not used right?
##########
File path: nifi-nar-bundles/nifi-extension-utils/nifi-hadoop-utils/src/main/java/org/apache/nifi/hadoop/SecurityUtil.java
##########
@@ -141,4 +143,37 @@ public static boolean isSecurityEnabled(final Configuration config) {
Validate.notNull(config);
return KERBEROS.equalsIgnoreCase(config.get(HADOOP_SECURITY_AUTHENTICATION));
}
+
+ public static <T> T callWithUgi(UserGroupInformation ugi, PrivilegedExceptionAction<T> action) throws IOException {
+ try {
+ return ugi.doAs(action);
+ } catch (InterruptedException e) {
+ throw new IOException(e);
+ }
+ }
+
+ public static void checkTGTAndRelogin(ComponentLog log, KerberosUser kerberosUser, UserGroupInformation ugi) throws IOException {
Review comment:
It looks like only the method below that takes log and kerberosUser is used, can we remove this one if it is unused?
##########
File path: nifi-nar-bundles/nifi-extension-utils/nifi-hadoop-utils/src/main/java/org/apache/nifi/hadoop/SecurityUtil.java
##########
@@ -141,4 +143,37 @@ public static boolean isSecurityEnabled(final Configuration config) {
Validate.notNull(config);
return KERBEROS.equalsIgnoreCase(config.get(HADOOP_SECURITY_AUTHENTICATION));
}
+
+ public static <T> T callWithUgi(UserGroupInformation ugi, PrivilegedExceptionAction<T> action) throws IOException {
Review comment:
It doesn't seem like we really need a utility method here just to call one line `return ugi.doAs(action);` , I think it would be simpler to just call `ugi.doAs` from all the places where this method is being called
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org