You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by Nixon Rodrigues <ni...@freestoneinfotech.com> on 2016/04/26 18:29:09 UTC
Review Request 46699: ATLAS-661 : Rest API Authentication.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46699/
-----------------------------------------------------------
Review request for atlas, Erik Bergenholtz, Shwetha GS, Selvamohan Neethiraj, Suma Shivaprasad, and Hemanth Yamijala.
Bugs: ATLAS-661
https://issues.apache.org/jira/browse/ATLAS-661
Repository: atlas
Description
-------
Introduced Rest API Authentication.
Introduced role and encoded password in user-credential file.
Authentication to fall back to file type if LDAP fails.
Changes in quick_start.py, atlas_admin.py & import_hive.sh to take password argument for basic authentication
Diffs
-----
addons/hive-bridge/src/bin/import-hive.sh 8df94f2
addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java 104c0c5
client/src/main/java/org/apache/atlas/AtlasAdminClient.java 473f72a
client/src/main/java/org/apache/atlas/AtlasClient.java be34802
dashboardv2/public/js/utils/Utils.js 7b2db79
distro/src/conf/users-credentials.properties 94e7cf4
webapp/src/main/java/org/apache/atlas/examples/QuickStart.java 79b8124
webapp/src/main/java/org/apache/atlas/web/dao/UserDao.java 76784c4
webapp/src/main/java/org/apache/atlas/web/filters/AtlasAuthenticationEntryPoint.java b77bd28
webapp/src/main/java/org/apache/atlas/web/security/AtlasAbstractAuthenticationProvider.java b2b11da
webapp/src/main/java/org/apache/atlas/web/security/AtlasAuthenticationProvider.java 7f87bd5
webapp/src/main/java/org/apache/atlas/web/security/AtlasFileAuthenticationProvider.java b3e3354
webapp/src/main/resources/spring-security.xml 8533473
webapp/src/main/webapp/WEB-INF/web.xml deb97d9
webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java 54d8d92
webapp/src/test/java/org/apache/atlas/web/security/FileAuthenticationTest.java 7e4c2c8
webapp/src/test/java/org/apache/atlas/web/security/UserDaoTest.java 5b5c70e
Diff: https://reviews.apache.org/r/46699/diff/
Testing
-------
Manual Testing done.
performed mvn clean install
performed mvn clean package -Pdist
Found Failed tests:
HiveLineageJerseyResourceIT.setUp:52->setupInstances:169->table:234->BaseResourceIT.createInstance:129 » AtlasService
Tests run: 10, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 22.476 sec <<< FAILURE! - in org.apache.atlas.repository.graph.GraphBackedRepositoryHardDeleteTest
testDisconnectBidirectionalReferences(org.apache.atlas.repository.graph.GraphBackedRepositoryHardDeleteTest) Time elapsed: 0.566 sec <<< FAILURE!
Thanks,
Nixon Rodrigues
Re: Review Request 46699: ATLAS-661 : Rest API Authentication.
Posted by Nixon Rodrigues <ni...@freestoneinfotech.com>.
> On April 27, 2016, 1:37 p.m., Shwetha GS wrote:
> > We currently support the following authentication methods:
> > 1. hadoop simple auth - assumes the username passed in the request. Doesn't do any authentication
> > 2. hadoop kerberos auth - using kerberos keytab
> > 3. simple file based authentication - username and password passed in the request, authenticated against file contents
> > 4. LDAP - username and password passed in the request, authenticated against LDAP server. If authentication fails, falls back to file based
> > 5. AD - username and password passed in the request, authenticated against AD server. If authentication fails, falls back to file based
> >
> > Assuming my above understanding is right, (correct me if I am wrong):
> >
> > For a single request, at both client and server, only one of the above authentication methods should be used. For example, requests coming from UI may use LDAP(with fall back to file based), requests from another source may use kerberos. There are couple of issues now:
> > 1. Currently, there are 2 authentication filters - AtlasAuthenticationProvider(handles 3,4, and 5 above) and AtlasAuthenticationFilter(handles 1&2 above, enabled with atlas.http.authentication.enabled=true). Both the filters will be enabled which means request will go through both which doesn't make sense. They need to be merged so that only one of the authentication methods is used for a request
> > 2. AtlasAuthenticationFilter sets user in RequestContext which is used for audit. Once the above 1st issue is sorted out, hopefully there is single place where authentication is done so that the request user is set from authenticated user
> > 3. AtlasClient now sets both username and password(used for file based/ldap/ad) and kerberos token(if kerberos enabled). There should be just one authentication mechanism for a request even at client
> > 4. With kerberos authentication, how do we get the authorities for the user
> >
> >
> > The utilities like quick_start, import-hive, atlas-admin etc can't always use 'admin' user. They should be allowed to run as the user running the command as long as they have authorisation to create entities API or to get admin status
Following points needs to be handled in separate JIRA.
1) Handling RequestContext for Audits
2) Authorization in secure environment (make sure that in Kerberos enabled environemnt Basic Auth is skipped).
- Nixon
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46699/#review130775
-----------------------------------------------------------
On May 6, 2016, 6:24 a.m., Nixon Rodrigues wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46699/
> -----------------------------------------------------------
>
> (Updated May 6, 2016, 6:24 a.m.)
>
>
> Review request for atlas, Erik Bergenholtz, Larry McCay, Madhan Neethiraj, Shwetha GS, Selvamohan Neethiraj, Suma Shivaprasad, Velmurugan Periasamy, and Hemanth Yamijala.
>
>
> Bugs: ATLAS-661
> https://issues.apache.org/jira/browse/ATLAS-661
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Introduced Rest API Authentication.
> Introduced role and encoded password in user-credential file.
> Authentication to fall back to file type if LDAP fails.
> Changes in quick_start.py, atlas_admin.py & import_hive.sh to take password argument for basic authentication
>
>
> Diffs
> -----
>
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java 104c0c5
> client/src/main/java/org/apache/atlas/AtlasAdminClient.java 473f72a
> client/src/main/java/org/apache/atlas/AtlasClient.java be34802
> common/src/main/java/org/apache/atlas/utils/AuthenticationUtil.java PRE-CREATION
> dashboardv2/public/js/utils/Utils.js 7b2db79
> distro/src/conf/users-credentials.properties 94e7cf4
> webapp/src/main/java/org/apache/atlas/examples/QuickStart.java 79b8124
> webapp/src/main/java/org/apache/atlas/web/dao/UserDao.java 76784c4
> webapp/src/main/java/org/apache/atlas/web/filters/AtlasAuthenticationEntryPoint.java b77bd28
> webapp/src/main/java/org/apache/atlas/web/security/AtlasADAuthenticationProvider.java 96dca45
> webapp/src/main/java/org/apache/atlas/web/security/AtlasAbstractAuthenticationProvider.java b2b11da
> webapp/src/main/java/org/apache/atlas/web/security/AtlasAuthenticationProvider.java 7f87bd5
> webapp/src/main/java/org/apache/atlas/web/security/AtlasFileAuthenticationProvider.java b3e3354
> webapp/src/main/java/org/apache/atlas/web/security/AtlasLdapAuthenticationProvider.java d911c1b
> webapp/src/main/resources/atlas-admin-site.xml ab8ba3d
> webapp/src/main/resources/spring-security.xml 8533473
> webapp/src/main/webapp/WEB-INF/web.xml deb97d9
> webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java 54d8d92
> webapp/src/test/java/org/apache/atlas/web/security/FileAuthenticationTest.java 7e4c2c8
> webapp/src/test/java/org/apache/atlas/web/security/NegativeSSLAndKerberosTest.java 9aac32a
> webapp/src/test/java/org/apache/atlas/web/security/SSLAndKerberosTest.java 2be52e7
> webapp/src/test/java/org/apache/atlas/web/security/UserDaoTest.java 5b5c70e
>
> Diff: https://reviews.apache.org/r/46699/diff/
>
>
> Testing
> -------
>
> Manual Testing done.
> performed mvn clean install
> performed mvn clean package -Pdist
>
> Found Failed tests:
> HiveLineageJerseyResourceIT.setUp:52->setupInstances:169->table:234->BaseResourceIT.createInstance:129 � AtlasService
>
> Tests run: 10, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 22.476 sec <<< FAILURE! - in org.apache.atlas.repository.graph.GraphBackedRepositoryHardDeleteTest
> testDisconnectBidirectionalReferences(org.apache.atlas.repository.graph.GraphBackedRepositoryHardDeleteTest) Time elapsed: 0.566 sec <<< FAILURE!
>
>
> Thanks,
>
> Nixon Rodrigues
>
>
Re: Review Request 46699: ATLAS-661 : Rest API Authentication.
Posted by Shwetha GS <ss...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46699/#review130775
-----------------------------------------------------------
We currently support the following authentication methods:
1. hadoop simple auth - assumes the username passed in the request. Doesn't do any authentication
2. hadoop kerberos auth - using kerberos keytab
3. simple file based authentication - username and password passed in the request, authenticated against file contents
4. LDAP - username and password passed in the request, authenticated against LDAP server. If authentication fails, falls back to file based
5. AD - username and password passed in the request, authenticated against AD server. If authentication fails, falls back to file based
Assuming my above understanding is right, (correct me if I am wrong):
For a single request, at both client and server, only one of the above authentication methods should be used. For example, requests coming from UI may use LDAP(with fall back to file based), requests from another source may use kerberos. There are couple of issues now:
1. Currently, there are 2 authentication filters - AtlasAuthenticationProvider(handles 3,4, and 5 above) and AtlasAuthenticationFilter(handles 1&2 above, enabled with atlas.http.authentication.enabled=true). Both the filters will be enabled which means request will go through both which doesn't make sense. They need to be merged so that only one of the authentication methods is used for a request
2. AtlasAuthenticationFilter sets user in RequestContext which is used for audit. Once the above 1st issue is sorted out, hopefully there is single place where authentication is done so that the request user is set from authenticated user
3. AtlasClient now sets both username and password(used for file based/ldap/ad) and kerberos token(if kerberos enabled). There should be just one authentication mechanism for a request even at client
4. With kerberos authentication, how do we get the authorities for the user
The utilities like quick_start, import-hive, atlas-admin etc can't always use 'admin' user. They should be allowed to run as the user running the command as long as they have authorisation to create entities API or to get admin status
webapp/src/main/java/org/apache/atlas/web/security/AtlasAbstractAuthenticationProvider.java (line 70)
<https://reviews.apache.org/r/46699/#comment194672>
Why is this hard-coded? This method shouldn't even be used. AtlasADAuthenticationProvider has DefaultLdapAuthoritiesPopulator, AtlasFileAuthenticationProvider loads from file etc
webapp/src/main/webapp/WEB-INF/web.xml
<https://reviews.apache.org/r/46699/#comment194673>
Please avoid moving lines without any changes. Makes it difficult to review
webapp/src/test/java/org/apache/atlas/web/security/UserDaoTest.java (line 33)
<https://reviews.apache.org/r/46699/#comment194676>
Add tests. There are no tests at all for the changes made
- Shwetha GS
On April 26, 2016, 4:29 p.m., Nixon Rodrigues wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46699/
> -----------------------------------------------------------
>
> (Updated April 26, 2016, 4:29 p.m.)
>
>
> Review request for atlas, Erik Bergenholtz, Shwetha GS, Selvamohan Neethiraj, Suma Shivaprasad, and Hemanth Yamijala.
>
>
> Bugs: ATLAS-661
> https://issues.apache.org/jira/browse/ATLAS-661
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Introduced Rest API Authentication.
> Introduced role and encoded password in user-credential file.
> Authentication to fall back to file type if LDAP fails.
> Changes in quick_start.py, atlas_admin.py & import_hive.sh to take password argument for basic authentication
>
>
> Diffs
> -----
>
> addons/hive-bridge/src/bin/import-hive.sh 8df94f2
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java 104c0c5
> client/src/main/java/org/apache/atlas/AtlasAdminClient.java 473f72a
> client/src/main/java/org/apache/atlas/AtlasClient.java be34802
> dashboardv2/public/js/utils/Utils.js 7b2db79
> distro/src/conf/users-credentials.properties 94e7cf4
> webapp/src/main/java/org/apache/atlas/examples/QuickStart.java 79b8124
> webapp/src/main/java/org/apache/atlas/web/dao/UserDao.java 76784c4
> webapp/src/main/java/org/apache/atlas/web/filters/AtlasAuthenticationEntryPoint.java b77bd28
> webapp/src/main/java/org/apache/atlas/web/security/AtlasAbstractAuthenticationProvider.java b2b11da
> webapp/src/main/java/org/apache/atlas/web/security/AtlasAuthenticationProvider.java 7f87bd5
> webapp/src/main/java/org/apache/atlas/web/security/AtlasFileAuthenticationProvider.java b3e3354
> webapp/src/main/resources/spring-security.xml 8533473
> webapp/src/main/webapp/WEB-INF/web.xml deb97d9
> webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java 54d8d92
> webapp/src/test/java/org/apache/atlas/web/security/FileAuthenticationTest.java 7e4c2c8
> webapp/src/test/java/org/apache/atlas/web/security/UserDaoTest.java 5b5c70e
>
> Diff: https://reviews.apache.org/r/46699/diff/
>
>
> Testing
> -------
>
> Manual Testing done.
> performed mvn clean install
> performed mvn clean package -Pdist
>
> Found Failed tests:
> HiveLineageJerseyResourceIT.setUp:52->setupInstances:169->table:234->BaseResourceIT.createInstance:129 » AtlasService
>
> Tests run: 10, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 22.476 sec <<< FAILURE! - in org.apache.atlas.repository.graph.GraphBackedRepositoryHardDeleteTest
> testDisconnectBidirectionalReferences(org.apache.atlas.repository.graph.GraphBackedRepositoryHardDeleteTest) Time elapsed: 0.566 sec <<< FAILURE!
>
>
> Thanks,
>
> Nixon Rodrigues
>
>
Re: Review Request 46699: ATLAS-661 : Rest API Authentication.
Posted by Nixon Rodrigues <ni...@freestoneinfotech.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46699/
-----------------------------------------------------------
(Updated May 5, 2016, 2:52 p.m.)
Review request for atlas, Erik Bergenholtz, Shwetha GS, Selvamohan Neethiraj, Suma Shivaprasad, and Hemanth Yamijala.
Changes
-------
Handled Shwetha's comments -- updated patch contains ::
1) Removal of default grantedAuthorities() method for LDAP & AD providor (in the LDAP and AD User / Group sync section).
2) Prompting username & password for client utilities such as quick_start.py, import-hive.sh & atlas_admin.py for basic api authentication.
AtlasClient changes to support basic authentication in non kerboros mode.
3) Added Test cases for user role mapping & validation for Filebased authentication.
**Testing Done**
Manually verifed Authentication feature
Ran mvn commands
Added more test cases to cover role mapping.
*NOTE: AD role check setting is verified & for LDAP role check is pending*
Bugs: ATLAS-661
https://issues.apache.org/jira/browse/ATLAS-661
Repository: atlas
Description
-------
Introduced Rest API Authentication.
Introduced role and encoded password in user-credential file.
Authentication to fall back to file type if LDAP fails.
Changes in quick_start.py, atlas_admin.py & import_hive.sh to take password argument for basic authentication
Diffs (updated)
-----
addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java 104c0c5
client/src/main/java/org/apache/atlas/AtlasAdminClient.java 473f72a
client/src/main/java/org/apache/atlas/AtlasClient.java be34802
common/src/main/java/org/apache/atlas/utils/AuthenticationUtil.java PRE-CREATION
dashboardv2/public/js/utils/Utils.js 7b2db79
distro/src/conf/users-credentials.properties 94e7cf4
webapp/src/main/java/org/apache/atlas/examples/QuickStart.java 79b8124
webapp/src/main/java/org/apache/atlas/web/dao/UserDao.java 76784c4
webapp/src/main/java/org/apache/atlas/web/filters/AtlasAuthenticationEntryPoint.java b77bd28
webapp/src/main/java/org/apache/atlas/web/security/AtlasADAuthenticationProvider.java 96dca45
webapp/src/main/java/org/apache/atlas/web/security/AtlasAbstractAuthenticationProvider.java b2b11da
webapp/src/main/java/org/apache/atlas/web/security/AtlasAuthenticationProvider.java 7f87bd5
webapp/src/main/java/org/apache/atlas/web/security/AtlasFileAuthenticationProvider.java b3e3354
webapp/src/main/java/org/apache/atlas/web/security/AtlasLdapAuthenticationProvider.java d911c1b
webapp/src/main/resources/atlas-admin-site.xml ab8ba3d
webapp/src/main/resources/spring-security.xml 8533473
webapp/src/main/webapp/WEB-INF/web.xml deb97d9
webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java 54d8d92
webapp/src/test/java/org/apache/atlas/web/security/FileAuthenticationTest.java 7e4c2c8
webapp/src/test/java/org/apache/atlas/web/security/NegativeSSLAndKerberosTest.java 9aac32a
webapp/src/test/java/org/apache/atlas/web/security/SSLAndKerberosTest.java 2be52e7
webapp/src/test/java/org/apache/atlas/web/security/UserDaoTest.java 5b5c70e
Diff: https://reviews.apache.org/r/46699/diff/
Testing
-------
Manual Testing done.
performed mvn clean install
performed mvn clean package -Pdist
Found Failed tests:
HiveLineageJerseyResourceIT.setUp:52->setupInstances:169->table:234->BaseResourceIT.createInstance:129 � AtlasService
Tests run: 10, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 22.476 sec <<< FAILURE! - in org.apache.atlas.repository.graph.GraphBackedRepositoryHardDeleteTest
testDisconnectBidirectionalReferences(org.apache.atlas.repository.graph.GraphBackedRepositoryHardDeleteTest) Time elapsed: 0.566 sec <<< FAILURE!
Thanks,
Nixon Rodrigues