You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@phoenix.apache.org by dbahir <gi...@git.apache.org> on 2016/08/29 15:58:09 UTC
[GitHub] phoenix pull request #203: [PHOENIX-3216] Kerberos ticket is not renewed whe...
GitHub user dbahir opened a pull request:
https://github.com/apache/phoenix/pull/203
[PHOENIX-3216] Kerberos ticket is not renewed when using Kerberos authentication with Phoenix JDBC driver
Kerberos ticket is not renewed when using Kerberos authentication with Phoenix JDBC driver. That is caused by UserInformationGroup loginUserFromKeytab being called multiple time from different threads if using a multi threaded environment. this fix ensures that there will only be one login per process.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/dbahir/phoenix master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/phoenix/pull/203.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #203
----
commit 37789fcfa1b322fac64bdebb9fb903313b3c1686
Author: Dan <db...@bloomberg.net>
Date: 2016-08-29T15:52:57Z
Ensure UGI's loginUserFromKeytab is only called once per JVM
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix issue #203: [PHOENIX-3216] Kerberos ticket is not renewed when using...
Posted by dbahir <gi...@git.apache.org>.
Github user dbahir commented on the issue:
https://github.com/apache/phoenix/pull/203
Closing, this issue will be fixed by [#191](https://github.com/apache/phoenix/pull/191)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix issue #203: [PHOENIX-3216] Kerberos ticket is not renewed when using...
Posted by dbahir <gi...@git.apache.org>.
Github user dbahir commented on the issue:
https://github.com/apache/phoenix/pull/203
This fix has been tested and it solves the issue, the same fix has been applied to the storm hdfs and hbase connectors.
https://issues.apache.org/jira/browse/STORM-1521
https://issues.apache.org/jira/browse/STORM-1535
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix issue #203: [PHOENIX-3216] Kerberos ticket is not renewed when using...
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on the issue:
https://github.com/apache/phoenix/pull/203
> That is caused by UserInformationGroup loginUserFromKeytab being called multiple time from different threads if using a multi threaded environment. this fix ensures that there will only be one login per process.
`UGI.loginUserFromKeytab` never spawns a renewal thread so as it is. I don't think this change has the effect you intend it to have.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix issue #203: [PHOENIX-3216] Kerberos ticket is not renewed when using...
Posted by dbahir <gi...@git.apache.org>.
Github user dbahir commented on the issue:
https://github.com/apache/phoenix/pull/203
If you look at https://github.com/hanborq/hadoop/blob/master/src/core/org/apache/hadoop/security/UserGroupInformation.java you can see that this class is not thread safe and not designed to have different users login in the same JVM as loginUser is defined in this way.
private static UserGroupInformation loginUser = null;
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix issue #203: [PHOENIX-3216] Kerberos ticket is not renewed when using...
Posted by dbahir <gi...@git.apache.org>.
Github user dbahir commented on the issue:
https://github.com/apache/phoenix/pull/203
Regarding the renewal, I understand from, http://stackoverflow.com/questions/34616676/should-i-call-ugi-checktgtandreloginfromkeytab-before-every-action-on-hadoop, that the RPC layer takes care of that.
I am trying to fix the scenario in which multiple threads call loginUserFromKeytab concurrently and then the renewal process no longer works as expected.
An example of that scenario is a storm topology that has multiple HBase/Phoenix/HDFS bolts in the same JVM. When the topology starts it will initialize all bolts which will execute a login from each one, when that happens the renewal no longer works. If only one login happens the renewal works properly.
In regarding to Phoenix, we came got into a similar situation with a multi-threaded application that caused loginUserFromKeytab to be called concurrently. The code change was made to protect that and works.
Your concern regarding security is correct.
I looked into PHOENIX-3189 which i was not aware of. The fix can be folded into it however we would need to handle synchronization of the loginUserFromKeytab if multple instances of the driver are created.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix issue #203: [PHOENIX-3216] Kerberos ticket is not renewed when using...
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on the issue:
https://github.com/apache/phoenix/pull/203
> Regarding the renewal, I understand from, http://stackoverflow.com/questions/34616676/should-i-call-ugi-checktgtandreloginfromkeytab-before-every-action-on-hadoop, that the RPC layer takes care of that.
Well, if you're talking to HDFS directly it would take care of it :). But we're talking about accessing HBase here. I'm not sure if the same holds true. I know there is something similar in the HBase RPC level, but I'd have to find it again in code to double check.
> I am trying to fix the scenario in which multiple threads call loginUserFromKeytab concurrently and then the renewal process no longer works as expected.
> If only one login happens the renewal works properly.
Is this the same principal over and over again? Are you essentially providing the same principal and keytab in the JDBC URL, expecting Phoenix to do everything for you instead of doing the login in Storm?
> Your concern regarding security is correct.
Ok. I would like to redirect your efforts to PHOENIX-3189 then. We cannot sacrifice security for multi-threading (as you can already handle the Kerberos login yourself). Can you take a look at the changes I have staged on #191? If this is the above case I outlined, we can add some concurrency control to prevent concurrent logins from happening.
> you can see that this class is not thread safe and not designed to have different users login in the same JVM as loginUser is defined in this way.
Phoenix itself is not well-designed to support concurrent (different) users accessing HBase because of how UGI works. If your application (Storm) needs to provide this functionality, Storm should perform logins itself, cache the UGI instances, and use {{UGI.doAs(..)}} instead of relying on the static state in UGI.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix issue #203: [PHOENIX-3216] Kerberos ticket is not renewed when using...
Posted by dbahir <gi...@git.apache.org>.
Github user dbahir commented on the issue:
https://github.com/apache/phoenix/pull/203
HBase renew implementation is similar to the HDFS one.
https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClientImpl.java#L658
Thanks for your comments, will look at your changes and see where these changes can fit in.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix issue #203: [PHOENIX-3216] Kerberos ticket is not renewed when using...
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on the issue:
https://github.com/apache/phoenix/pull/203
> This fix has been tested and it solves the issue, the same fix has been applied to the storm hdfs and hbase connectors.
But I still don't understand what you're trying to fix. https://github.com/apache/hadoop/blob/94225152399e6e89fa7b4cff6d17d33e544329a3/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java#L957-L958
`UserGroupInformation` does *not* spawn any renewal thread for ticket renewal. Can you clarify what doesn't work? Given your description on JIRA, it doesn't make sense to me.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix issue #203: [PHOENIX-3216] Kerberos ticket is not renewed when using...
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on the issue:
https://github.com/apache/phoenix/pull/203
Ignoring the aforementioned issue, I don't think this change is correctly handling multiple users.
It would be re-introducing the bug that was talked about in PHOENIX-3126. If there was a user that was already logged in and then a different URL was provided with different credentials, the old user's credentials would be used instead of the new user's credentials. This would be a security vulnerability.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request #203: [PHOENIX-3216] Kerberos ticket is not renewed whe...
Posted by dbahir <gi...@git.apache.org>.
Github user dbahir closed the pull request at:
https://github.com/apache/phoenix/pull/203
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---