You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by lordjc <gi...@git.apache.org> on 2016/03/02 20:57:15 UTC
[GitHub] nifi pull request: NIFI-1488 Added hbase kerb auth with ugi
GitHub user lordjc opened a pull request:
https://github.com/apache/nifi/pull/253
NIFI-1488 Added hbase kerb auth with ugi
Added kerberos authentication functionality for hbase service.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/lordjc/nifi hbase-kerb
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/253.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #253
----
commit 99fb0a393472d60a81214afdd4c5cae268b77227
Author: Jeff Lord <jl...@cloudera.com>
Date: 2016-03-02T19:35:10Z
Added hbase kerb auth with ugi
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1488 Added hbase kerb auth with ugi
Posted by alopresto <gi...@git.apache.org>.
Github user alopresto commented on the pull request:
https://github.com/apache/nifi/pull/253#issuecomment-194034813
I looked at the Hadoop source code a little bit, but without getting too deep, it does appear that it could change during the lifetime of an application:
1. Hadoop/HBase is configured with Kerberos
2. NiFi is started and HBase processor is added and configured to use Kerberos to connect (processor properties not saved yet, perhaps?)
3. HBase is no longer using Kerberos
4. NiFi HBase processor `customValidate` is run and uses "cached" value of `isSecurityEnabled() == true`, even though Kerberos is no longer valid for this client
That's why I think the `UserGroupInformation.isSecurityEnabled()` call should still be executed during the `customValidate` call of the processor. To me, that seems "closer to true" than the alternative.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1488 Added hbase kerb auth with ugi
Posted by rickysaltzer <gi...@git.apache.org>.
Github user rickysaltzer commented on the pull request:
https://github.com/apache/nifi/pull/253#issuecomment-193912884
I was able to build and test this from scratch on a new machine. Looks like the travis-ci error is regarding the RELP processor, so not related to this patch.
I ran some functional tests against an HBase cluster with Kerberos enabled. Built a simple pipeline to ingest JSON events, and everything worked as expected. Awesome contribution @lordjc!
After testing, I feel comfortable accepting this patch without implementing a thick unit test with a MiniKDC. @alopresto do you have any additional comments or concerns before I commit this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1488 Added hbase kerb auth with ugi
Posted by rickysaltzer <gi...@git.apache.org>.
Github user rickysaltzer commented on the pull request:
https://github.com/apache/nifi/pull/253#issuecomment-193944624
@bbende yeah, I noticed that all of the HBase tests were using a mock pattern. I think we should create a new JIRA to convert HBase tests to use a mini-hbase cluster. Doing so in this patch would be a little out of scope considering this is just adding Kerberos support.
I think adding a couple tests for Kerberos options is reasonable, though.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1488 Added hbase kerb auth with ugi
Posted by rickysaltzer <gi...@git.apache.org>.
Github user rickysaltzer commented on a diff in the pull request:
https://github.com/apache/nifi/pull/253#discussion_r54787370
--- Diff: nifi-nar-bundles/nifi-standard-services/nifi-hbase_1_1_2-client-service-bundle/nifi-hbase_1_1_2-client-service/src/main/java/org/apache/nifi/hbase/HBase_1_1_2_ClientService.java ---
@@ -171,6 +188,15 @@ protected Connection createConnection(final ConfigurationContext context) throws
}
}
+ UserGroupInformation.setConfiguration(hbaseConfig);
+
+ if (UserGroupInformation.isSecurityEnabled()) {
+ LOG.info("SECURITY IS ENABLED");
+ UserGroupInformation.loginUserFromKeytab(context.getProperty(KERBEROS_PRINCIPAL).getValue(),
--- End diff --
Can we try/catch IOException when logging in the user in case of an exception. It can be as simple as logging the error being with the keytab and then bubble up the exception.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1488 Added hbase kerb auth with ugi
Posted by bbende <gi...@git.apache.org>.
Github user bbende commented on the pull request:
https://github.com/apache/nifi/pull/253#issuecomment-193920825
@rickysaltzer if you rebase to master I think the RELP failures should go away.
Regarding testing, a couple of thoughts...
1) Could we update the validation test to include the new logic for Kerberos?
https://github.com/apache/nifi/blob/40dd8a0a845ef5f4d4fde451f02376ab2fab9758/nifi-nar-bundles/nifi-standard-services/nifi-hbase_1_1_2-client-service-bundle/nifi-hbase_1_1_2-client-service/src/test/java/org/apache/nifi/hbase/TestHBase_1_1_2_ClientService.java#L61
2) There are also a bunch of tests in there that mock everything, not sure how feasible this is, but is it possible to add a test where we somehow mocked the static calls to UserGroupInformation to at least prove that the correct values are getting passed when security is enabled?
I'm not that familiar with mocking static methods so not sure what this entails.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1488 Added hbase kerb auth with ugi
Posted by lordjc <gi...@git.apache.org>.
Github user lordjc closed the pull request at:
https://github.com/apache/nifi/pull/253
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1488 Added hbase kerb auth with ugi
Posted by rickysaltzer <gi...@git.apache.org>.
Github user rickysaltzer commented on a diff in the pull request:
https://github.com/apache/nifi/pull/253#discussion_r54796523
--- Diff: nifi-nar-bundles/nifi-standard-services/nifi-hbase_1_1_2-client-service-bundle/nifi-hbase_1_1_2-client-service/src/main/java/org/apache/nifi/hbase/HBase_1_1_2_ClientService.java ---
@@ -171,6 +188,15 @@ protected Connection createConnection(final ConfigurationContext context) throws
}
}
+ UserGroupInformation.setConfiguration(hbaseConfig);
+
+ if (UserGroupInformation.isSecurityEnabled()) {
+ LOG.info("SECURITY IS ENABLED");
+ UserGroupInformation.loginUserFromKeytab(context.getProperty(KERBEROS_PRINCIPAL).getValue(),
--- End diff --
After the initial login, you'll need to periodically renew your ticket. See [AbstractHadoopProcessor.java](https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/AbstractHadoopProcessor.java#L425-L439) for an example.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1488 Added hbase kerb auth with ugi
Posted by rickysaltzer <gi...@git.apache.org>.
Github user rickysaltzer commented on a diff in the pull request:
https://github.com/apache/nifi/pull/253#discussion_r54946428
--- Diff: nifi-nar-bundles/nifi-standard-services/nifi-hbase_1_1_2-client-service-bundle/nifi-hbase_1_1_2-client-service/src/main/java/org/apache/nifi/hbase/HBase_1_1_2_ClientService.java ---
@@ -171,6 +188,15 @@ protected Connection createConnection(final ConfigurationContext context) throws
}
}
+ UserGroupInformation.setConfiguration(hbaseConfig);
+
+ if (UserGroupInformation.isSecurityEnabled()) {
+ LOG.info("SECURITY IS ENABLED");
+ UserGroupInformation.loginUserFromKeytab(context.getProperty(KERBEROS_PRINCIPAL).getValue(),
--- End diff --
@lordjc has a good point. Running a constant ingestion pipeline overnight should help validate your suspicion.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1488 Added hbase kerb auth with ugi
Posted by rickysaltzer <gi...@git.apache.org>.
Github user rickysaltzer commented on the pull request:
https://github.com/apache/nifi/pull/253#issuecomment-194484021
just built on my local machine, as well. I'll go ahead and rebase and then push to `master` today.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1488 Added hbase kerb auth with ugi
Posted by rickysaltzer <gi...@git.apache.org>.
Github user rickysaltzer commented on a diff in the pull request:
https://github.com/apache/nifi/pull/253#discussion_r54787487
--- Diff: nifi-nar-bundles/nifi-standard-services/nifi-hbase_1_1_2-client-service-bundle/nifi-hbase_1_1_2-client-service/src/main/java/org/apache/nifi/hbase/HBase_1_1_2_ClientService.java ---
@@ -171,6 +188,15 @@ protected Connection createConnection(final ConfigurationContext context) throws
}
}
+ UserGroupInformation.setConfiguration(hbaseConfig);
+
+ if (UserGroupInformation.isSecurityEnabled()) {
+ LOG.info("SECURITY IS ENABLED");
--- End diff --
Preferably a little more information, such as `HBase Security Enabled, Logging in as User {}`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1488 Added hbase kerb auth with ugi
Posted by alopresto <gi...@git.apache.org>.
Github user alopresto commented on the pull request:
https://github.com/apache/nifi/pull/253#issuecomment-194465114
@lordjc if we are confident the result of `UserGroupInformation.isSecurityEnabled()` does not change between class instantiation and `customValidate()` execution, I am ok with this PR as is. Thanks for the contribution.
_Note:_ there are test failures on Travis CI for `java.lang.OutOfMemoryError: GC overhead limit exceeded` and test runner max time exceeded. These may just be Travis issues; does the full build with tests run successfully on your local system?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1488 Added hbase kerb auth with ugi
Posted by lordjc <gi...@git.apache.org>.
Github user lordjc commented on the pull request:
https://github.com/apache/nifi/pull/253#issuecomment-199314205
Closing based on comments above.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1488 Added hbase kerb auth with ugi
Posted by lordjc <gi...@git.apache.org>.
Github user lordjc commented on a diff in the pull request:
https://github.com/apache/nifi/pull/253#discussion_r54946384
--- Diff: nifi-nar-bundles/nifi-standard-services/nifi-hbase_1_1_2-client-service-bundle/nifi-hbase_1_1_2-client-service/src/main/java/org/apache/nifi/hbase/HBase_1_1_2_ClientService.java ---
@@ -171,6 +188,15 @@ protected Connection createConnection(final ConfigurationContext context) throws
}
}
+ UserGroupInformation.setConfiguration(hbaseConfig);
+
+ if (UserGroupInformation.isSecurityEnabled()) {
+ LOG.info("SECURITY IS ENABLED");
+ UserGroupInformation.loginUserFromKeytab(context.getProperty(KERBEROS_PRINCIPAL).getValue(),
--- End diff --
Link to spawnAutoRenewalThreadForUserCreds()
https://github.com/apache/hadoop/blob/release-2.7.1/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java#L872
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1488 Added hbase kerb auth with ugi
Posted by rickysaltzer <gi...@git.apache.org>.
Github user rickysaltzer commented on the pull request:
https://github.com/apache/nifi/pull/253#issuecomment-194465576
@alopresto I'm rebuilding now from scratch.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1488 Added hbase kerb auth with ugi
Posted by alopresto <gi...@git.apache.org>.
Github user alopresto commented on the pull request:
https://github.com/apache/nifi/pull/253#issuecomment-194012946
@rickysaltzer Yes, I did not mean full-stack integration tests with a KDC, just unit tests, primarily focusing on validation, especially of user input and retrieved credentials, as that is a common security attack vector.
I do appreciate you adding simple validation tests. My only question is regarding the extraction of the boolean check for `UserGroupInformation.isSecurityEnabled()` in [`HBase_1_1_2_ClientService.java`](https://github.com/rickysaltzer/nifi/blob/6dc8416b7f2cb5480a6f6ae63777ea4080263e67/nifi-nar-bundles/nifi-standard-services/nifi-hbase_1_1_2-client-service-bundle/nifi-hbase_1_1_2-client-service/src/main/java/org/apache/nifi/hbase/HBase_1_1_2_ClientService.java) on line 137. I'm not completely familiar with the HBase components, but my understanding is that this will now be executed statically at the class initialization time, rather than during the `customValidate` method call. Is this intentional?
As for @bbende , I think Groovy tests allow for easier static mocking, but I do not consider it a priority for this PR. I think the existing unit tests @rickysaltzer added are sufficient (although perhaps with a perfunctory check of the error message/cause after the `runner.assertNotValid(service);` just to ensure it's not accidentally testing something else.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1488 Added hbase kerb auth with ugi
Posted by alopresto <gi...@git.apache.org>.
Github user alopresto commented on a diff in the pull request:
https://github.com/apache/nifi/pull/253#discussion_r54791243
--- Diff: nifi-nar-bundles/nifi-standard-services/nifi-hbase-client-service-api/src/main/java/org/apache/nifi/hbase/HBaseClientService.java ---
@@ -40,6 +40,18 @@
.addValidator(new ConfigFilesValidator())
.build();
+ PropertyDescriptor KERBEROS_PRINCIPAL = new PropertyDescriptor.Builder()
+ .name("Kerberos Principal")
+ .description("Principal of user writing to hbase")
+ .addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+ .build();
+
+ PropertyDescriptor KERBEROS_KEYTAB = new PropertyDescriptor.Builder()
+ .name("Kerberos Keytab")
--- End diff --
For new `PropertyDescriptor`s, please use a unique permanent identifier string of the form `kerberos-keytab` for the `name` property and use the `displayName` property to hold UI values (can change in the future, internationalization, etc.). The `name` property is used for object resolution, and changing the value in the future to respond to user feedback will break backward compatibility with existing flows.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1488 Added hbase kerb auth with ugi
Posted by rickysaltzer <gi...@git.apache.org>.
Github user rickysaltzer commented on a diff in the pull request:
https://github.com/apache/nifi/pull/253#discussion_r54787538
--- Diff: nifi-nar-bundles/nifi-standard-services/nifi-hbase_1_1_2-client-service-bundle/nifi-hbase_1_1_2-client-service/src/main/java/org/apache/nifi/hbase/HBase_1_1_2_ClientService.java ---
@@ -171,6 +188,15 @@ protected Connection createConnection(final ConfigurationContext context) throws
}
}
+ UserGroupInformation.setConfiguration(hbaseConfig);
+
+ if (UserGroupInformation.isSecurityEnabled()) {
+ LOG.info("SECURITY IS ENABLED");
+ UserGroupInformation.loginUserFromKeytab(context.getProperty(KERBEROS_PRINCIPAL).getValue(),
+ context.getProperty(KERBEROS_KEYTAB).getValue());
+ } else {
+ LOG.info("SIMPLE AUTHENTICATION");
--- End diff --
less ambiguous
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1488 Added hbase kerb auth with ugi
Posted by joewitt <gi...@git.apache.org>.
Github user joewitt commented on the pull request:
https://github.com/apache/nifi/pull/253#issuecomment-198146996
Hello @lordjc This ended up being merged and incorporated but the 'This closes #253' was not added to the commit log so the close hook didn't fire. Can you please close this PR?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1488 Added hbase kerb auth with ugi
Posted by lordjc <gi...@git.apache.org>.
Github user lordjc commented on the pull request:
https://github.com/apache/nifi/pull/253#issuecomment-194448347
@alopresto
Thank you for the feedback.
Turning kerberos on and off for a cluster is a non trivial task, but if that did in fact occur one would need to update all of the configuration files.
"NiFi is started and HBase processor is added and configured to use Kerberos to connect (processor properties not saved yet, perhaps?)"
This is not a processor but a "service" which gets enabled. In order to enable it you need to first save the service config. At this point the ugi.isSecurityEnabled() method would be run and subsequently the validations would be run to include customValidate.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1488 Added hbase kerb auth with ugi
Posted by rickysaltzer <gi...@git.apache.org>.
Github user rickysaltzer commented on the pull request:
https://github.com/apache/nifi/pull/253#issuecomment-193964547
@lordjc I've added a few Kerberos options to the service validation tests. I've also rebased the changes on `master`. You can grab the changes from https://github.com/rickysaltzer/nifi/tree/lord-kerberos.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1488 Added hbase kerb auth with ugi
Posted by lordjc <gi...@git.apache.org>.
Github user lordjc commented on the pull request:
https://github.com/apache/nifi/pull/253#issuecomment-194475452
It builds with tests successfully on my local system.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1488 Added hbase kerb auth with ugi
Posted by alopresto <gi...@git.apache.org>.
Github user alopresto commented on the pull request:
https://github.com/apache/nifi/pull/253#issuecomment-191431327
Have you written any unit or integration tests for this new functionality?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1488 Added hbase kerb auth with ugi
Posted by alopresto <gi...@git.apache.org>.
Github user alopresto commented on a diff in the pull request:
https://github.com/apache/nifi/pull/253#discussion_r54791214
--- Diff: nifi-nar-bundles/nifi-standard-services/nifi-hbase-client-service-api/src/main/java/org/apache/nifi/hbase/HBaseClientService.java ---
@@ -40,6 +40,18 @@
.addValidator(new ConfigFilesValidator())
.build();
+ PropertyDescriptor KERBEROS_PRINCIPAL = new PropertyDescriptor.Builder()
+ .name("Kerberos Principal")
--- End diff --
For new `PropertyDescriptor`s, please use a unique permanent identifier string of the form `kerberos-principal` for the `name` property and use the `displayName` property to hold UI values (can change in the future, internationalization, etc.). The `name` property is used for object resolution, and changing the value in the future to respond to user feedback will break backward compatibility with existing flows.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1488 Added hbase kerb auth with ugi
Posted by rickysaltzer <gi...@git.apache.org>.
Github user rickysaltzer commented on the pull request:
https://github.com/apache/nifi/pull/253#issuecomment-194026785
@alopresto The `UserGroupInformation` is all referenced statically, and before my changes the `UserGroupInformation.isSecurityEnabled()` method was called at the time of `customValidate`. I'm not familiar enough with the way `UserGroupInformation` works to know whether not `isSecurityEnabled()` can change throughout the lifetime of a NiFi process. I added the change in order to override that boolean during tests. @lordjc, do you know?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1488 Added hbase kerb auth with ugi
Posted by lordjc <gi...@git.apache.org>.
Github user lordjc commented on a diff in the pull request:
https://github.com/apache/nifi/pull/253#discussion_r54945555
--- Diff: nifi-nar-bundles/nifi-standard-services/nifi-hbase_1_1_2-client-service-bundle/nifi-hbase_1_1_2-client-service/src/main/java/org/apache/nifi/hbase/HBase_1_1_2_ClientService.java ---
@@ -171,6 +188,15 @@ protected Connection createConnection(final ConfigurationContext context) throws
}
}
+ UserGroupInformation.setConfiguration(hbaseConfig);
+
+ if (UserGroupInformation.isSecurityEnabled()) {
+ LOG.info("SECURITY IS ENABLED");
+ UserGroupInformation.loginUserFromKeytab(context.getProperty(KERBEROS_PRINCIPAL).getValue(),
--- End diff --
Ok so looking at UGI it seems to spawn a kerberos credential renewal thread. I think that maybe will not work for processors because of thread suspension. However this is not a processor it is a service. Does anyone know if "services" once enabled ever suspend threads? If not than we may be safe letting UGI handle the renewal of kerberos credentials.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1488 Added hbase kerb auth with ugi
Posted by jackowaya <gi...@git.apache.org>.
Github user jackowaya commented on the pull request:
https://github.com/apache/nifi/pull/253#issuecomment-191977983
The indispensable Hadoop and Kerberos: The Madness Beyond the Gate claims this can be done using MiniKDC https://steveloughran.gitbooks.io/kerberos_and_hadoop/content/sections/testing.html
It also describes it as the hardest form of testing and gives no example. Does anyone have experience with it or a good example to follow? I think it isn't in Nifi anywhere for the existing Kerberos interactions.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1488 Added hbase kerb auth with ugi
Posted by rickysaltzer <gi...@git.apache.org>.
Github user rickysaltzer commented on a diff in the pull request:
https://github.com/apache/nifi/pull/253#discussion_r54787933
--- Diff: nifi-nar-bundles/nifi-standard-services/nifi-hbase-client-service-api/src/main/java/org/apache/nifi/hbase/HBaseClientService.java ---
@@ -40,6 +40,18 @@
.addValidator(new ConfigFilesValidator())
.build();
+ PropertyDescriptor KERBEROS_PRINCIPAL = new PropertyDescriptor.Builder()
+ .name("Kerberos Principal")
+ .description("Principal of user writing to hbase")
+ .addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
--- End diff --
I believe using the `StandardValidators.NON_EMPTY_VALIDATOR` will cause the processor to be invalid if we want to talk to a non-secure HBase cluster, since our principal would be blank/empty.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request: NIFI-1488 Added hbase kerb auth with ugi
Posted by rickysaltzer <gi...@git.apache.org>.
Github user rickysaltzer commented on the pull request:
https://github.com/apache/nifi/pull/253#issuecomment-194504062
rebased and pushed to `master`. Thanks again, @lordjc.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---