You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by John Lilley <jo...@redpoint.net> on 2015/08/17 17:28:26 UTC
UserGroupInformation and login with password
Greetings,
Our software uses UserGroupInformation to authenticate with Kerberos-secure clusters. We've found that there are obvious methods for logging in via keytab:
loginUserFromKeytab()
reloginFromKeytab()
However, there are not obvious analogous methods for password-based login. We've created the equivalent to loginUserFromPassword() using reflection to access private members, but have not yet created the equivalent reloginFromPassword().
It doesn't seem right to be using reflection here, but we cannot find the public API for principal/password login and relogin. It seems like this should be something simple. We do need to support password, because many of our customers do not allow keytabs.
Thanks
John Lilley
RE: UserGroupInformation and login with password
Posted by John Lilley <jo...@redpoint.net>.
I don't see how to do this in our architecture. Our clients may be across VPNs and completely inaccessible to the KDC. Our server basically functions as the Hadoop client, even though it is a long-running service on the OS host. When a user logs in, it performs these steps:
- Spawn a new process using CreateProcessWithLogon on Windows, or fork/PAM/setuid on Linux.
- Use UsergroupInformation to login the user using the login/password and obtain a ticket for the UGI's default user context.
All of this works just fine, although we are forced to use reflection to gain access to the private setLogin() method of UGI.
The thing is, we cannot simply login again in the same process to avoid ticket expiration. It doesn't work. I see that there is dedicated code in reloginFromKeytab() to clear out the old tokens and install new ones while keeping the UGI.class locked. We need to be able to do something similar given the login and password.
It would be great if there were an in-process API that completely mimicked kinit and managed the TGT cache instead of the in-memory data of UGI. But we haven't found one. And we can't just run kinit, because it has no method to inject a password (assume intentionally, since there isn't a way to secure that as a command-line argument).
As I mentioned before, we can get tickets using login/password via UGI. This works. But I get lost around "how does one re-login and use the new ticket?". Here's what we observe:
- We do a login and see the TGT in the UGI
- After accessing HDFS, we see the TGT and the HDFS ticket in UGI.
- After user.logout(), both tickets disappear
- After a new login, a new TGT appears
- Accessing HDFS fails and no HDFS ticket is added.
We are trying to understand why this process diverges from reloginFromKeytab(), which works just fine.
John Lilley
From: Zheng, Kai [mailto:kai.zheng@intel.com]
Sent: Monday, August 17, 2015 5:40 PM
To: user@hadoop.apache.org
Subject: RE: UserGroupInformation and login with password
Hi John,
Login from keytab is mostly expected for services. For end users, yes they use passwords. In Kerberos (and Hadoop), it's expected for end users to execute kinit like tool and generate ticket caches, then some methods like login from ticket cache in UGI will do the left work and help in your case.
Or do you have to use the password directly in your program? If so, you may add the method by yourself: 1) let your program prompt to user for password; 2) if your program has gathered the password in other means, then use some support like below:
In Krb5LoginModule:
* useFirstPass if, true, this LoginModule retrieves the
* username and password from the module's shared state,
* using "javax.security.auth.login.name" and
* "javax.security.auth.login.password" as the respective
* keys. The retrieved values are used for authentication.
* If authentication fails, no attempt for a retry
* is made, and the failure is reported back to the
* calling application.
Hope this helps.
Regards,
Kai
From: John Lilley [mailto:john.lilley@redpoint.net]
Sent: Monday, August 17, 2015 11:28 PM
To: 'user@hadoop.apache.org'
Subject: UserGroupInformation and login with password
Greetings,
Our software uses UserGroupInformation to authenticate with Kerberos-secure clusters. We've found that there are obvious methods for logging in via keytab:
loginUserFromKeytab()
reloginFromKeytab()
However, there are not obvious analogous methods for password-based login. We've created the equivalent to loginUserFromPassword() using reflection to access private members, but have not yet created the equivalent reloginFromPassword().
It doesn't seem right to be using reflection here, but we cannot find the public API for principal/password login and relogin. It seems like this should be something simple. We do need to support password, because many of our customers do not allow keytabs.
Thanks
John Lilley
RE: UserGroupInformation and login with password
Posted by John Lilley <jo...@redpoint.net>.
I don't see how to do this in our architecture. Our clients may be across VPNs and completely inaccessible to the KDC. Our server basically functions as the Hadoop client, even though it is a long-running service on the OS host. When a user logs in, it performs these steps:
- Spawn a new process using CreateProcessWithLogon on Windows, or fork/PAM/setuid on Linux.
- Use UsergroupInformation to login the user using the login/password and obtain a ticket for the UGI's default user context.
All of this works just fine, although we are forced to use reflection to gain access to the private setLogin() method of UGI.
The thing is, we cannot simply login again in the same process to avoid ticket expiration. It doesn't work. I see that there is dedicated code in reloginFromKeytab() to clear out the old tokens and install new ones while keeping the UGI.class locked. We need to be able to do something similar given the login and password.
It would be great if there were an in-process API that completely mimicked kinit and managed the TGT cache instead of the in-memory data of UGI. But we haven't found one. And we can't just run kinit, because it has no method to inject a password (assume intentionally, since there isn't a way to secure that as a command-line argument).
As I mentioned before, we can get tickets using login/password via UGI. This works. But I get lost around "how does one re-login and use the new ticket?". Here's what we observe:
- We do a login and see the TGT in the UGI
- After accessing HDFS, we see the TGT and the HDFS ticket in UGI.
- After user.logout(), both tickets disappear
- After a new login, a new TGT appears
- Accessing HDFS fails and no HDFS ticket is added.
We are trying to understand why this process diverges from reloginFromKeytab(), which works just fine.
John Lilley
From: Zheng, Kai [mailto:kai.zheng@intel.com]
Sent: Monday, August 17, 2015 5:40 PM
To: user@hadoop.apache.org
Subject: RE: UserGroupInformation and login with password
Hi John,
Login from keytab is mostly expected for services. For end users, yes they use passwords. In Kerberos (and Hadoop), it's expected for end users to execute kinit like tool and generate ticket caches, then some methods like login from ticket cache in UGI will do the left work and help in your case.
Or do you have to use the password directly in your program? If so, you may add the method by yourself: 1) let your program prompt to user for password; 2) if your program has gathered the password in other means, then use some support like below:
In Krb5LoginModule:
* useFirstPass if, true, this LoginModule retrieves the
* username and password from the module's shared state,
* using "javax.security.auth.login.name" and
* "javax.security.auth.login.password" as the respective
* keys. The retrieved values are used for authentication.
* If authentication fails, no attempt for a retry
* is made, and the failure is reported back to the
* calling application.
Hope this helps.
Regards,
Kai
From: John Lilley [mailto:john.lilley@redpoint.net]
Sent: Monday, August 17, 2015 11:28 PM
To: 'user@hadoop.apache.org'
Subject: UserGroupInformation and login with password
Greetings,
Our software uses UserGroupInformation to authenticate with Kerberos-secure clusters. We've found that there are obvious methods for logging in via keytab:
loginUserFromKeytab()
reloginFromKeytab()
However, there are not obvious analogous methods for password-based login. We've created the equivalent to loginUserFromPassword() using reflection to access private members, but have not yet created the equivalent reloginFromPassword().
It doesn't seem right to be using reflection here, but we cannot find the public API for principal/password login and relogin. It seems like this should be something simple. We do need to support password, because many of our customers do not allow keytabs.
Thanks
John Lilley
RE: UserGroupInformation and login with password
Posted by John Lilley <jo...@redpoint.net>.
I don't see how to do this in our architecture. Our clients may be across VPNs and completely inaccessible to the KDC. Our server basically functions as the Hadoop client, even though it is a long-running service on the OS host. When a user logs in, it performs these steps:
- Spawn a new process using CreateProcessWithLogon on Windows, or fork/PAM/setuid on Linux.
- Use UsergroupInformation to login the user using the login/password and obtain a ticket for the UGI's default user context.
All of this works just fine, although we are forced to use reflection to gain access to the private setLogin() method of UGI.
The thing is, we cannot simply login again in the same process to avoid ticket expiration. It doesn't work. I see that there is dedicated code in reloginFromKeytab() to clear out the old tokens and install new ones while keeping the UGI.class locked. We need to be able to do something similar given the login and password.
It would be great if there were an in-process API that completely mimicked kinit and managed the TGT cache instead of the in-memory data of UGI. But we haven't found one. And we can't just run kinit, because it has no method to inject a password (assume intentionally, since there isn't a way to secure that as a command-line argument).
As I mentioned before, we can get tickets using login/password via UGI. This works. But I get lost around "how does one re-login and use the new ticket?". Here's what we observe:
- We do a login and see the TGT in the UGI
- After accessing HDFS, we see the TGT and the HDFS ticket in UGI.
- After user.logout(), both tickets disappear
- After a new login, a new TGT appears
- Accessing HDFS fails and no HDFS ticket is added.
We are trying to understand why this process diverges from reloginFromKeytab(), which works just fine.
John Lilley
From: Zheng, Kai [mailto:kai.zheng@intel.com]
Sent: Monday, August 17, 2015 5:40 PM
To: user@hadoop.apache.org
Subject: RE: UserGroupInformation and login with password
Hi John,
Login from keytab is mostly expected for services. For end users, yes they use passwords. In Kerberos (and Hadoop), it's expected for end users to execute kinit like tool and generate ticket caches, then some methods like login from ticket cache in UGI will do the left work and help in your case.
Or do you have to use the password directly in your program? If so, you may add the method by yourself: 1) let your program prompt to user for password; 2) if your program has gathered the password in other means, then use some support like below:
In Krb5LoginModule:
* useFirstPass if, true, this LoginModule retrieves the
* username and password from the module's shared state,
* using "javax.security.auth.login.name" and
* "javax.security.auth.login.password" as the respective
* keys. The retrieved values are used for authentication.
* If authentication fails, no attempt for a retry
* is made, and the failure is reported back to the
* calling application.
Hope this helps.
Regards,
Kai
From: John Lilley [mailto:john.lilley@redpoint.net]
Sent: Monday, August 17, 2015 11:28 PM
To: 'user@hadoop.apache.org'
Subject: UserGroupInformation and login with password
Greetings,
Our software uses UserGroupInformation to authenticate with Kerberos-secure clusters. We've found that there are obvious methods for logging in via keytab:
loginUserFromKeytab()
reloginFromKeytab()
However, there are not obvious analogous methods for password-based login. We've created the equivalent to loginUserFromPassword() using reflection to access private members, but have not yet created the equivalent reloginFromPassword().
It doesn't seem right to be using reflection here, but we cannot find the public API for principal/password login and relogin. It seems like this should be something simple. We do need to support password, because many of our customers do not allow keytabs.
Thanks
John Lilley
RE: UserGroupInformation and login with password
Posted by John Lilley <jo...@redpoint.net>.
I don't see how to do this in our architecture. Our clients may be across VPNs and completely inaccessible to the KDC. Our server basically functions as the Hadoop client, even though it is a long-running service on the OS host. When a user logs in, it performs these steps:
- Spawn a new process using CreateProcessWithLogon on Windows, or fork/PAM/setuid on Linux.
- Use UsergroupInformation to login the user using the login/password and obtain a ticket for the UGI's default user context.
All of this works just fine, although we are forced to use reflection to gain access to the private setLogin() method of UGI.
The thing is, we cannot simply login again in the same process to avoid ticket expiration. It doesn't work. I see that there is dedicated code in reloginFromKeytab() to clear out the old tokens and install new ones while keeping the UGI.class locked. We need to be able to do something similar given the login and password.
It would be great if there were an in-process API that completely mimicked kinit and managed the TGT cache instead of the in-memory data of UGI. But we haven't found one. And we can't just run kinit, because it has no method to inject a password (assume intentionally, since there isn't a way to secure that as a command-line argument).
As I mentioned before, we can get tickets using login/password via UGI. This works. But I get lost around "how does one re-login and use the new ticket?". Here's what we observe:
- We do a login and see the TGT in the UGI
- After accessing HDFS, we see the TGT and the HDFS ticket in UGI.
- After user.logout(), both tickets disappear
- After a new login, a new TGT appears
- Accessing HDFS fails and no HDFS ticket is added.
We are trying to understand why this process diverges from reloginFromKeytab(), which works just fine.
John Lilley
From: Zheng, Kai [mailto:kai.zheng@intel.com]
Sent: Monday, August 17, 2015 5:40 PM
To: user@hadoop.apache.org
Subject: RE: UserGroupInformation and login with password
Hi John,
Login from keytab is mostly expected for services. For end users, yes they use passwords. In Kerberos (and Hadoop), it's expected for end users to execute kinit like tool and generate ticket caches, then some methods like login from ticket cache in UGI will do the left work and help in your case.
Or do you have to use the password directly in your program? If so, you may add the method by yourself: 1) let your program prompt to user for password; 2) if your program has gathered the password in other means, then use some support like below:
In Krb5LoginModule:
* useFirstPass if, true, this LoginModule retrieves the
* username and password from the module's shared state,
* using "javax.security.auth.login.name" and
* "javax.security.auth.login.password" as the respective
* keys. The retrieved values are used for authentication.
* If authentication fails, no attempt for a retry
* is made, and the failure is reported back to the
* calling application.
Hope this helps.
Regards,
Kai
From: John Lilley [mailto:john.lilley@redpoint.net]
Sent: Monday, August 17, 2015 11:28 PM
To: 'user@hadoop.apache.org'
Subject: UserGroupInformation and login with password
Greetings,
Our software uses UserGroupInformation to authenticate with Kerberos-secure clusters. We've found that there are obvious methods for logging in via keytab:
loginUserFromKeytab()
reloginFromKeytab()
However, there are not obvious analogous methods for password-based login. We've created the equivalent to loginUserFromPassword() using reflection to access private members, but have not yet created the equivalent reloginFromPassword().
It doesn't seem right to be using reflection here, but we cannot find the public API for principal/password login and relogin. It seems like this should be something simple. We do need to support password, because many of our customers do not allow keytabs.
Thanks
John Lilley
RE: UserGroupInformation and login with password
Posted by "Zheng, Kai" <ka...@intel.com>.
Hi John,
Login from keytab is mostly expected for services. For end users, yes they use passwords. In Kerberos (and Hadoop), it's expected for end users to execute kinit like tool and generate ticket caches, then some methods like login from ticket cache in UGI will do the left work and help in your case.
Or do you have to use the password directly in your program? If so, you may add the method by yourself: 1) let your program prompt to user for password; 2) if your program has gathered the password in other means, then use some support like below:
In Krb5LoginModule:
* useFirstPass if, true, this LoginModule retrieves the
* username and password from the module's shared state,
* using "javax.security.auth.login.name" and
* "javax.security.auth.login.password" as the respective
* keys. The retrieved values are used for authentication.
* If authentication fails, no attempt for a retry
* is made, and the failure is reported back to the
* calling application.
Hope this helps.
Regards,
Kai
From: John Lilley [mailto:john.lilley@redpoint.net]
Sent: Monday, August 17, 2015 11:28 PM
To: 'user@hadoop.apache.org'
Subject: UserGroupInformation and login with password
Greetings,
Our software uses UserGroupInformation to authenticate with Kerberos-secure clusters. We've found that there are obvious methods for logging in via keytab:
loginUserFromKeytab()
reloginFromKeytab()
However, there are not obvious analogous methods for password-based login. We've created the equivalent to loginUserFromPassword() using reflection to access private members, but have not yet created the equivalent reloginFromPassword().
It doesn't seem right to be using reflection here, but we cannot find the public API for principal/password login and relogin. It seems like this should be something simple. We do need to support password, because many of our customers do not allow keytabs.
Thanks
John Lilley
RE: UserGroupInformation and login with password
Posted by "Zheng, Kai" <ka...@intel.com>.
Hi John,
Login from keytab is mostly expected for services. For end users, yes they use passwords. In Kerberos (and Hadoop), it's expected for end users to execute kinit like tool and generate ticket caches, then some methods like login from ticket cache in UGI will do the left work and help in your case.
Or do you have to use the password directly in your program? If so, you may add the method by yourself: 1) let your program prompt to user for password; 2) if your program has gathered the password in other means, then use some support like below:
In Krb5LoginModule:
* useFirstPass if, true, this LoginModule retrieves the
* username and password from the module's shared state,
* using "javax.security.auth.login.name" and
* "javax.security.auth.login.password" as the respective
* keys. The retrieved values are used for authentication.
* If authentication fails, no attempt for a retry
* is made, and the failure is reported back to the
* calling application.
Hope this helps.
Regards,
Kai
From: John Lilley [mailto:john.lilley@redpoint.net]
Sent: Monday, August 17, 2015 11:28 PM
To: 'user@hadoop.apache.org'
Subject: UserGroupInformation and login with password
Greetings,
Our software uses UserGroupInformation to authenticate with Kerberos-secure clusters. We've found that there are obvious methods for logging in via keytab:
loginUserFromKeytab()
reloginFromKeytab()
However, there are not obvious analogous methods for password-based login. We've created the equivalent to loginUserFromPassword() using reflection to access private members, but have not yet created the equivalent reloginFromPassword().
It doesn't seem right to be using reflection here, but we cannot find the public API for principal/password login and relogin. It seems like this should be something simple. We do need to support password, because many of our customers do not allow keytabs.
Thanks
John Lilley
RE: UserGroupInformation and login with password
Posted by "Zheng, Kai" <ka...@intel.com>.
Hi John,
Login from keytab is mostly expected for services. For end users, yes they use passwords. In Kerberos (and Hadoop), it's expected for end users to execute kinit like tool and generate ticket caches, then some methods like login from ticket cache in UGI will do the left work and help in your case.
Or do you have to use the password directly in your program? If so, you may add the method by yourself: 1) let your program prompt to user for password; 2) if your program has gathered the password in other means, then use some support like below:
In Krb5LoginModule:
* useFirstPass if, true, this LoginModule retrieves the
* username and password from the module's shared state,
* using "javax.security.auth.login.name" and
* "javax.security.auth.login.password" as the respective
* keys. The retrieved values are used for authentication.
* If authentication fails, no attempt for a retry
* is made, and the failure is reported back to the
* calling application.
Hope this helps.
Regards,
Kai
From: John Lilley [mailto:john.lilley@redpoint.net]
Sent: Monday, August 17, 2015 11:28 PM
To: 'user@hadoop.apache.org'
Subject: UserGroupInformation and login with password
Greetings,
Our software uses UserGroupInformation to authenticate with Kerberos-secure clusters. We've found that there are obvious methods for logging in via keytab:
loginUserFromKeytab()
reloginFromKeytab()
However, there are not obvious analogous methods for password-based login. We've created the equivalent to loginUserFromPassword() using reflection to access private members, but have not yet created the equivalent reloginFromPassword().
It doesn't seem right to be using reflection here, but we cannot find the public API for principal/password login and relogin. It seems like this should be something simple. We do need to support password, because many of our customers do not allow keytabs.
Thanks
John Lilley
RE: UserGroupInformation and login with password
Posted by "Zheng, Kai" <ka...@intel.com>.
Hi John,
Login from keytab is mostly expected for services. For end users, yes they use passwords. In Kerberos (and Hadoop), it's expected for end users to execute kinit like tool and generate ticket caches, then some methods like login from ticket cache in UGI will do the left work and help in your case.
Or do you have to use the password directly in your program? If so, you may add the method by yourself: 1) let your program prompt to user for password; 2) if your program has gathered the password in other means, then use some support like below:
In Krb5LoginModule:
* useFirstPass if, true, this LoginModule retrieves the
* username and password from the module's shared state,
* using "javax.security.auth.login.name" and
* "javax.security.auth.login.password" as the respective
* keys. The retrieved values are used for authentication.
* If authentication fails, no attempt for a retry
* is made, and the failure is reported back to the
* calling application.
Hope this helps.
Regards,
Kai
From: John Lilley [mailto:john.lilley@redpoint.net]
Sent: Monday, August 17, 2015 11:28 PM
To: 'user@hadoop.apache.org'
Subject: UserGroupInformation and login with password
Greetings,
Our software uses UserGroupInformation to authenticate with Kerberos-secure clusters. We've found that there are obvious methods for logging in via keytab:
loginUserFromKeytab()
reloginFromKeytab()
However, there are not obvious analogous methods for password-based login. We've created the equivalent to loginUserFromPassword() using reflection to access private members, but have not yet created the equivalent reloginFromPassword().
It doesn't seem right to be using reflection here, but we cannot find the public API for principal/password login and relogin. It seems like this should be something simple. We do need to support password, because many of our customers do not allow keytabs.
Thanks
John Lilley