You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ranger.apache.org by me...@apache.org on 2018/12/19 11:50:50 UTC

[ranger] branch ranger-1.1 updated (f96b8a7 -> 44c4a3d)

This is an automated email from the ASF dual-hosted git repository.

mehul pushed a change to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git.


    from f96b8a7  RANGER-2308: User role user should not able to access usersync audit report if it does not have permissions on the audit module.
     new f4a6a45  RANGER-2210:Ranger support for Apache Kafka 2.0.0
     new b3c9600  RANGER-2231 - Upgrade to Knox 1.1.0
     new c6fe231  RANGER-2239 - Update to surefire 2.21.0
     new 4d12157  RANGER-2228: Updated docs for Apache Ranger 1.2.0 release
     new 78064a2  RANGER-2222:Apache RangerKafkaPlugin support to handle Kafka Cluster as a new resource
     new 861876d  RANGER-2222:Apache RangerKafkaPlugin support to handle Kafka Cluster as a new resource - added create permission in Cluster resource
     new 6af25a7  RANGER-1958 [HBase] Implement getUserPermissions API of AccessControlService.Interface to allow clients to access HBase permissions stored in Ranger (Ankit Singhal)
     new f5f7f33  RANGER-2237: Upgrade Kylin version to 2.5.0
     new fbcdac0  RANGER-2252:Permission Kafka Admin should not be part of Topic resource in Ranger Kafka resource definition
     new 8561502  RANGER-2256 - Grammatical error in UI
     new eeec458  RANGER-2243: Provide option to ranger builds to specifically build a single plugin
     new dccd0dc  RANGER-2263: Removed unnecessary explicit dependency for apache commons compress jar in Ranger
     new 282f2fd  RANGER-2258: Improve the policy list page to prompt users when the service is disabled
     new ea8df62  RANGER-2264:Kafka default policies for new resources are not showing up in UI when upgrade is done from older version
     new 41e0b90  RANGER-2257:Add policyID to error message when click the Access log of Audit
     new 156f48f  RANGER-2248: Sorting does not work in AbstractPredicateUtil.java
     new ec0c3b4  RANGER-2265: Added all ranger modules to linux profile so that all are built by default for unix environments
     new 2d0d8e7  RANGER-2266:To make Id to ID in Audit Pages of Ranger Admin
     new 32144cc  RANGER-2280:The emptyText of User Sync and Plugin Status should be reasonable
     new 45392da  RANGER-2277: Kylin repository config missing 'Common Name for Certificate'
     new 90a3877  RANGER-2267: Add a icon to differentiate the status of the service
     new 96936b9  RANGER-2049: Added support for doAs for Ranger REST APIs with Kerberized mode
     new b9f6986  RANGER-2049: Fixed an issue where doAs User role is not set properly
     new aefc2b3  RANGER-2276:Email Address should be verified when Add New User in Ranger Admin
     new 187d8e8  RANGER-2284: Unable to build image using docker
     new 6ec3f99  RANGER-2288: Sqoop repository config missing 'Common Name for Certificate'
     new 1b08671  RANGER-2282: The error message for changing password is incorrect in User Profile page.
     new 1f8e788  RANGER-2289: Unable to get Audit Admin tab page
     new 7cbfd8b  RANGER-2244: Tomcat Security Vulnerability Alert. The version of the tomcat for ranger should upgrade to 7.0.91 or later.
     new 5b60229  RANGER-2292 : Test case fix for RANGER-2276
     new 16157f6  RANGER-2298 Modify JAVA_VERSION_REQUIRED to 1.8 in install.properties
     new c5ba1ce  RANGER-2303:Add kylin-plugin infomation to README.txt
     new 7e7649a  RANGER-2299 Modify the permissions of the kms install.properties file to 700
     new 4dc2fda  RANGER-2294:Front-end and back-end email address regular expression should be the same
     new 9d07e83  RANGER-2163:Spelling error in the PatchPersmissionModel_J10003.java
     new d57e363  RANGER-2306 : Add support for X-Forwarded-for header in Knox plugin
     new af6d186  RANGER-2307: Better error message, and a NULL check for the native code
     new 6ecd4fb  RANGER-2295: Set specific Ranger version in patches status entry table
     new 44c4a3d  Updating year in NOTICE.txt

The 39 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 NOTICE.txt                                         |   2 +-
 README.txt                                         |   2 +
 .../ranger/plugin/store/AbstractPredicateUtil.java |   2 +-
 .../service-defs/ranger-servicedef-kafka.json      |  49 +-
 .../service-defs/ranger-servicedef-kylin.json      |  11 +
 .../service-defs/ranger-servicedef-sqoop.json      |  13 +-
 build_ranger_using_docker.sh                       |  20 +-
 docs/pom.xml                                       | 108 ++++
 docs/src/site/site.xml                             |   1 +
 docs/src/site/xdoc/download.xml                    |  12 +-
 .../ranger/server/tomcat/EmbeddedServer.java       |   4 -
 .../authorization/hbase/AuthorizationSession.java  |   3 +-
 .../hbase/RangerAuthorizationCoprocessor.java      | 122 ++++-
 .../hbase/HBaseRangerAuthorizationTest.java        |  71 ++-
 hbase-agent/src/test/resources/hbase-policies.json |  58 +++
 kms/scripts/install.properties                     |   2 +-
 .../authorization/knox/KnoxRangerPlugin.java       |  13 +
 .../authorization/knox/RangerPDPKnoxFilter.java    |  26 +-
 .../kafka/authorizer/RangerKafkaAuditHandler.java  |  74 +++
 .../kafka/authorizer/RangerKafkaAuthorizer.java    |  24 +-
 .../authorizer/KafkaRangerAuthorizerGSSTest.java   |   1 -
 .../authorizer/KafkaRangerAuthorizerTest.java      |   6 +-
 .../authorizer/KafkaRangerTopicCreationTest.java   | 191 +++++++
 .../src/test/resources/kafka-policies.json         | 198 ++++++-
 .../src/test/resources/kafka_kerberos.jaas         |   8 +-
 pom.xml                                            | 566 ++++++++++++++++++---
 .../optimized/current/ranger_core_db_mysql.sql     |   1 +
 .../optimized/current/ranger_core_db_oracle.sql    |   1 +
 .../optimized/current/ranger_core_db_postgres.sql  |   1 +
 .../current/ranger_core_db_sqlanywhere.sql         |   2 +
 .../optimized/current/ranger_core_db_sqlserver.sql |   1 +
 security-admin/pom.xml                             |  12 +-
 security-admin/scripts/db_setup.py                 |  66 ++-
 security-admin/scripts/install.properties          |   2 +-
 .../java/org/apache/ranger/biz/ServiceDBStore.java |   2 +-
 .../main/java/org/apache/ranger/biz/UserMgr.java   |   4 +-
 .../main/java/org/apache/ranger/biz/XUserMgr.java  |  33 +-
 .../PatchForKafkaServiceDefUpdate_J10025.java      | 448 ++++++++++++++++
 ...10003.java => PatchPermissionModel_J10003.java} |   8 +-
 .../web/filter/RangerKRBAuthenticationFilter.java  | 132 ++++-
 .../webapp/scripts/modules/globalize/message/en.js |   9 +-
 .../src/main/webapp/scripts/utils/XAUtils.js       |  17 +-
 .../views/policies/RangerPolicyTableLayout.js      |   4 +-
 .../webapp/scripts/views/reports/AuditLayout.js    |  16 +-
 .../main/webapp/scripts/views/user/UserProfile.js  |   2 +-
 .../main/webapp/scripts/views/users/UserForm.js    |   6 +-
 .../src/main/webapp/templates/helpers/XAHelpers.js |   7 +-
 .../policies/RangerPolicyTableLayout_tmpl.html     |   2 +-
 .../java/org/apache/ranger/biz/TestXUserMgr.java   |   2 +
 src/main/assembly/kms.xml                          |   9 +-
 src/main/assembly/plugin-kafka.xml                 |   1 -
 src/main/assembly/tagsync.xml                      |   2 +-
 unixauthnative/src/main/c/credValidator.c          |   7 +-
 unixauthpam/src/main/c/pamCredValidator.c          |   2 +-
 54 files changed, 2157 insertions(+), 229 deletions(-)
 create mode 100644 plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuditHandler.java
 create mode 100644 plugin-kafka/src/test/java/org/apache/ranger/authorization/kafka/authorizer/KafkaRangerTopicCreationTest.java
 create mode 100644 security-admin/src/main/java/org/apache/ranger/patch/PatchForKafkaServiceDefUpdate_J10025.java
 rename security-admin/src/main/java/org/apache/ranger/patch/{PatchPersmissionModel_J10003.java => PatchPermissionModel_J10003.java} (96%)


[ranger] 06/39: RANGER-2222:Apache RangerKafkaPlugin support to handle Kafka Cluster as a new resource - added create permission in Cluster resource

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit 861876d69aa8c552e70edcf4cfcf99bd9fdd0ccc
Author: rmani <rm...@hortonworks.com>
AuthorDate: Mon Oct 8 18:11:40 2018 -0700

    RANGER-2222:Apache RangerKafkaPlugin support to handle Kafka Cluster as a new resource - added create permission in Cluster resource
---
 .../src/main/resources/service-defs/ranger-servicedef-kafka.json        | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/agents-common/src/main/resources/service-defs/ranger-servicedef-kafka.json b/agents-common/src/main/resources/service-defs/ranger-servicedef-kafka.json
index 7e91aab..78ae9ea 100644
--- a/agents-common/src/main/resources/service-defs/ranger-servicedef-kafka.json
+++ b/agents-common/src/main/resources/service-defs/ranger-servicedef-kafka.json
@@ -56,7 +56,7 @@
 			},
 			"label":"Cluster",
 			"description":"Cluster",
-			"accessTypeRestrictions": ["configure", "alter_configs", "describe", "describe_configs", "kafka_admin", "idempotent_write"]
+			"accessTypeRestrictions": ["create", "configure", "alter_configs", "describe", "describe_configs", "kafka_admin", "idempotent_write"]
 		},
 		{
 			"itemId":4,


[ranger] 31/39: RANGER-2298 Modify JAVA_VERSION_REQUIRED to 1.8 in install.properties

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit 16157f69c40ebd409caf794902c6795059d90622
Author: zhangqiang2 <zh...@zte.com.cn>
AuthorDate: Fri Nov 30 15:27:34 2018 +0800

    RANGER-2298 Modify JAVA_VERSION_REQUIRED to 1.8 in install.properties
    
    Signed-off-by: peng.jianhua <pe...@zte.com.cn>
---
 kms/scripts/install.properties            | 2 +-
 security-admin/scripts/install.properties | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/kms/scripts/install.properties b/kms/scripts/install.properties
index 947d9f3..aea0bb8 100755
--- a/kms/scripts/install.properties
+++ b/kms/scripts/install.properties
@@ -230,7 +230,7 @@ TMPFILE=$PWD/.fi_tmp
 LOGFILE=$PWD/logfile
 
 JAVA_BIN='java'
-JAVA_VERSION_REQUIRED='1.7'
+JAVA_VERSION_REQUIRED='1.8'
 JAVA_ORACLE='Java(TM) SE Runtime Environment'
 
 mysql_core_file=db/mysql/kms_core_db.sql
diff --git a/security-admin/scripts/install.properties b/security-admin/scripts/install.properties
index 674844c..fdcee1b 100644
--- a/security-admin/scripts/install.properties
+++ b/security-admin/scripts/install.properties
@@ -223,7 +223,7 @@ LOGFILE=$PWD/logfile
 LOGFILES="$LOGFILE"
 
 JAVA_BIN='java'
-JAVA_VERSION_REQUIRED='1.7'
+JAVA_VERSION_REQUIRED='1.8'
 JAVA_ORACLE='Java(TM) SE Runtime Environment'
 
 ranger_admin_max_heap_size=1g


[ranger] 10/39: RANGER-2256 - Grammatical error in UI

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit 856150298d3aac0a61e4eda7216e6d29faebda44
Author: Colm O hEigeartaigh <co...@apache.org>
AuthorDate: Wed Oct 17 17:19:52 2018 +0100

    RANGER-2256 - Grammatical error in UI
    
    Signed-off-by: Colm O hEigeartaigh <co...@apache.org>
---
 security-admin/src/main/webapp/scripts/modules/globalize/message/en.js | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/security-admin/src/main/webapp/scripts/modules/globalize/message/en.js b/security-admin/src/main/webapp/scripts/modules/globalize/message/en.js
index 99cbf55..19cc7b4 100644
--- a/security-admin/src/main/webapp/scripts/modules/globalize/message/en.js
+++ b/security-admin/src/main/webapp/scripts/modules/globalize/message/en.js
@@ -434,7 +434,7 @@ define(function(require) {
                 pleaseSelectAccessTypeForTagMasking : 'Please select access type first to enable add masking options.',
                 addUserOrGroupForDelegateAdmin      : 'Please select user/group for the selected permission(s)',
                 policyLabelsinfo		: 'Enter label of policy',
-                noUserFoundText			: 'No user associate with this group.',
+                noUserFoundText			: 'No user is associated with this group.',
                 showInitialHundredUser  : 'Initially search filter is applied for first hundred users. To get more users click on ',
                 searchForUserSync       :"Search for your user sync audits...",
                 policyExpired           :'Policy Expired',


[ranger] 27/39: RANGER-2282: The error message for changing password is incorrect in User Profile page.

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit 1b08671e87acfd37ae40daef47a53c96c91e318a
Author: zhangqiang2 <zh...@zte.com.cn>
AuthorDate: Tue Nov 13 10:48:04 2018 +0800

    RANGER-2282: The error message for changing password is incorrect in User Profile page.
    
    Signed-off-by: peng.jianhua <pe...@zte.com.cn>
---
 security-admin/src/main/java/org/apache/ranger/biz/UserMgr.java  | 4 ++--
 security-admin/src/main/webapp/scripts/views/user/UserProfile.js | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/security-admin/src/main/java/org/apache/ranger/biz/UserMgr.java b/security-admin/src/main/java/org/apache/ranger/biz/UserMgr.java
index 2a638f8..9e45782 100644
--- a/security-admin/src/main/java/org/apache/ranger/biz/UserMgr.java
+++ b/security-admin/src/main/java/org/apache/ranger/biz/UserMgr.java
@@ -418,7 +418,7 @@ public class UserMgr {
 		String encryptedOldPwd = encrypt(pwdChange.getLoginId(),pwdChange.getOldPassword());
 		if (!stringUtil.equals(encryptedOldPwd, gjUser.getPassword())) {
 			logger.info("changePassword(). Invalid old password. LoginId="+ pwdChange.getLoginId());
-			throw restErrorUtil.createRESTException("serverMsg.userMgrOldPassword",MessageEnums.INVALID_INPUT_DATA, null, null,pwdChange.getLoginId());
+			throw restErrorUtil.createRESTException("validationMessages.oldPasswordError",MessageEnums.INVALID_INPUT_DATA, null, null,pwdChange.getLoginId());
 		}
 
 		//validate new password
@@ -448,7 +448,7 @@ public class UserMgr {
 		} else {
 			ret.setMsgDesc("Password update failed");
 			ret.setStatusCode(VXResponse.STATUS_ERROR);
-			throw restErrorUtil.createRESTException("serverMsg.userMgrNewPassword",MessageEnums.INVALID_INPUT_DATA, gjUser.getId(),"password", gjUser.toString());
+			throw restErrorUtil.createRESTException("serverMsg.userMgrOldPassword",MessageEnums.INVALID_INPUT_DATA, gjUser.getId(),"password", gjUser.toString());
 		}
 		return ret;
 	}
diff --git a/security-admin/src/main/webapp/scripts/views/user/UserProfile.js b/security-admin/src/main/webapp/scripts/views/user/UserProfile.js
index 8e60b53..408b311 100644
--- a/security-admin/src/main/webapp/scripts/views/user/UserProfile.js
+++ b/security-admin/src/main/webapp/scripts/views/user/UserProfile.js
@@ -171,7 +171,7 @@ define(function(require){
 						that.form.fields.newPassword.setError(localization.tt('validationMessages.newPasswordError'));
 						that.form.fields.reEnterPassword.setError(localization.tt('validationMessages.newPasswordError'));
 					}else if((msResponse.responseJSON.msgDesc) == "serverMsg.userMgrOldPassword"){
-						that.form.fields.oldPassword.setError(localization.tt('validationMessages.oldPasswordRepeatError'));
+						that.form.fields.newPassword.setError(localization.tt('validationMessages.oldPasswordRepeatError'));
                                         }else if(msResponse.status == 419){
                                                 XAUtil.defaultErrorHandler(options , msResponse);
 					} else {


[ranger] 24/39: RANGER-2276:Email Address should be verified when Add New User in Ranger Admin

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit aefc2b3be06fb76bf4866c2307311d3a1e316a5d
Author: zhangqiang2 <zh...@zte.com.cn>
AuthorDate: Mon Nov 12 16:31:17 2018 +0800

    RANGER-2276:Email Address should be verified when Add New User in Ranger Admin
    
    Signed-off-by: peng.jianhua <pe...@zte.com.cn>
---
 .../main/java/org/apache/ranger/biz/XUserMgr.java  | 33 ++++++++++++++--------
 1 file changed, 21 insertions(+), 12 deletions(-)

diff --git a/security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java b/security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java
index a875db6..e330b5a 100644
--- a/security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java
+++ b/security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java
@@ -30,6 +30,7 @@ import java.util.Objects;
 import java.util.Set;
 
 import org.apache.commons.collections.CollectionUtils;
+import org.apache.commons.lang.StringUtils;
 import org.apache.ranger.common.ContextUtil;
 import org.apache.ranger.common.GUIDUtil;
 import org.apache.ranger.common.RangerCommonEnums;
@@ -44,13 +45,13 @@ import org.apache.ranger.security.context.RangerAPIMapping;
 import org.apache.ranger.service.*;
 import org.apache.ranger.view.*;
 import org.apache.log4j.Logger;
-import org.apache.ranger.authorization.utils.StringUtil;
 import org.apache.ranger.common.AppConstants;
 import org.apache.ranger.common.MessageEnums;
 import org.apache.ranger.common.PropertiesUtil;
 import org.apache.ranger.common.RangerConstants;
 import org.apache.ranger.common.RangerServicePoliciesCache;
 import org.apache.ranger.common.SearchCriteria;
+import org.apache.ranger.common.StringUtil;
 import org.apache.ranger.common.UserSessionBase;
 import org.apache.ranger.db.RangerDaoManager;
 import org.apache.ranger.db.XXAuditMapDao;
@@ -85,7 +86,6 @@ import org.springframework.transaction.annotation.Transactional;
 import javax.servlet.http.HttpServletResponse;
 
 import org.apache.ranger.entity.XXPortalUserRole;
-import org.springframework.util.StringUtils;
 
 @Component
 public class XUserMgr extends XUserMgrBase {
@@ -141,9 +141,10 @@ public class XUserMgr extends XUserMgrBase {
 	@Autowired
 	XGroupUserService xGroupUserService;
 
-	static final Logger logger = Logger.getLogger(XUserMgr.class);
-
+	@Autowired
+	StringUtil stringUtil;
 
+	static final Logger logger = Logger.getLogger(XUserMgr.class);
 
 	public VXUser getXUserByUserName(String userName) {
 		VXUser vXUser=null;
@@ -191,7 +192,15 @@ public class XUserMgr extends XUserMgrBase {
 		if("null".equalsIgnoreCase(vXPortalUser.getLastName())){
 			vXPortalUser.setLastName("");
 		}
-		vXPortalUser.setEmailAddress(vXUser.getEmailAddress());
+
+		String emailAddress = vXUser.getEmailAddress();
+		if (StringUtils.isNotEmpty(emailAddress) && !stringUtil.validateEmail(emailAddress)) {
+			logger.warn("Invalid email address:" + emailAddress);
+			throw restErrorUtil.createRESTException("Please provide valid email address.",
+					MessageEnums.INVALID_INPUT_DATA);
+		}
+		vXPortalUser.setEmailAddress(emailAddress);
+
 		if (vXPortalUser.getFirstName() != null
 				&& vXPortalUser.getLastName() != null
 				&& !vXPortalUser.getFirstName().trim().isEmpty()
@@ -1065,7 +1074,7 @@ public class XUserMgr extends XUserMgrBase {
 		List<VXUserPermission> userPermListOld = new ArrayList<VXUserPermission>();
 
 		XXModuleDef xModuleDef = daoManager.getXXModuleDef().getById(vXModuleDef.getId());
-		if(!StringUtil.equals(xModuleDef.getModule(), vXModuleDef.getModule())) {
+		if(!StringUtils.equals(xModuleDef.getModule(), vXModuleDef.getModule())) {
 			throw restErrorUtil.createRESTException("Module name change is not allowed!", MessageEnums.DATA_NOT_UPDATABLE);
 		}
 		VXModuleDef vModuleDefPopulateOld = xModuleDefService.populateViewBean(xModuleDef);
@@ -1979,7 +1988,7 @@ public class XUserMgr extends XUserMgrBase {
 		XXGroupDao xXGroupDao = daoManager.getXXGroup();
 		XXGroup xXGroup = xXGroupDao.getById(id);
 		VXGroup vXGroup = xGroupService.populateViewBean(xXGroup);
-		if (vXGroup == null || StringUtil.isEmpty(vXGroup.getName())) {
+		if (vXGroup == null || StringUtils.isEmpty(vXGroup.getName())) {
 			throw restErrorUtil.createRESTException("Group ID doesn't exist.", MessageEnums.INVALID_INPUT_DATA);
 		}
 		if(logger.isDebugEnabled()){
@@ -2148,7 +2157,7 @@ public class XUserMgr extends XUserMgrBase {
 		XXUserDao xXUserDao = daoManager.getXXUser();
 		XXUser xXUser =	xXUserDao.getById(id);
 		VXUser vXUser =	xUserService.populateViewBean(xXUser);
-		if(vXUser==null ||StringUtil.isEmpty(vXUser.getName())){
+		if(vXUser==null || StringUtils.isEmpty(vXUser.getName())){
 			throw restErrorUtil.createRESTException("No user found with id=" + id);
 		}
 		XXPortalUserDao xXPortalUserDao=daoManager.getXXPortalUser();
@@ -2157,7 +2166,7 @@ public class XUserMgr extends XUserMgrBase {
 		if(xXPortalUser!=null){
 			vXPortalUser=xPortalUserService.populateViewBean(xXPortalUser);
 		}
-		if(vXPortalUser==null ||StringUtil.isEmpty(vXPortalUser.getLoginId())){
+		if(vXPortalUser==null || StringUtils.isEmpty(vXPortalUser.getLoginId())){
 			throw restErrorUtil.createRESTException("No user found with id=" + id);
 		}
 		if (logger.isDebugEnabled()) {
@@ -2331,10 +2340,10 @@ public class XUserMgr extends XUserMgrBase {
 	private <T extends RangerPolicyItem> void removeUserGroupReferences(List<T> policyItems, String user, String group) {
 		List<T> itemsToRemove = null;
 		for(T policyItem : policyItems) {
-			if(!StringUtil.isEmpty(user)) {
+			if(StringUtils.isNotEmpty(user)) {
 				policyItem.getUsers().remove(user);
 			}
-			if(!StringUtil.isEmpty(group)) {
+			if(StringUtils.isNotEmpty(group)) {
 				policyItem.getGroups().remove(group);
 			}
 			if(policyItem.getUsers().isEmpty() && policyItem.getGroups().isEmpty()) {
@@ -2355,7 +2364,7 @@ public class XUserMgr extends XUserMgrBase {
 			if (!session.isUserAdmin()) {
 				throw restErrorUtil.create403RESTException("Operation denied. LoggedInUser= "+session.getXXPortalUser().getLoginId() + " isn't permitted to perform the action.");
 			}else{
-				if(!StringUtil.isEmpty(loginID) && loginID.equals(session.getLoginId())){
+				if(StringUtils.isNotEmpty(loginID) && loginID.equals(session.getLoginId())){
 					throw restErrorUtil.create403RESTException("Operation denied. LoggedInUser= "+session.getXXPortalUser().getLoginId() + " isn't permitted to delete his own profile.");
 				}
 			}


[ranger] 28/39: RANGER-2289: Unable to get Audit Admin tab page

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit 1f8e788ad626c9739bad2ef8b416dafce6f2088b
Author: zhangqiang2 <zh...@zte.com.cn>
AuthorDate: Tue Nov 20 17:43:07 2018 +0800

    RANGER-2289: Unable to get Audit Admin tab page
    
    Signed-off-by: peng.jianhua <pe...@zte.com.cn>
---
 security-admin/src/main/webapp/scripts/utils/XAUtils.js | 17 ++++++++---------
 .../main/webapp/scripts/views/reports/AuditLayout.js    |  4 ++--
 2 files changed, 10 insertions(+), 11 deletions(-)

diff --git a/security-admin/src/main/webapp/scripts/utils/XAUtils.js b/security-admin/src/main/webapp/scripts/utils/XAUtils.js
index bb88ec3..d85dc7a 100644
--- a/security-admin/src/main/webapp/scripts/utils/XAUtils.js
+++ b/security-admin/src/main/webapp/scripts/utils/XAUtils.js
@@ -770,15 +770,14 @@ define(function(require) {
 					text : m.attributes.category
 				});
 				var extraParam = {};
-				if (_.has(serverParamName, 'multiple')
-						&& serverParamName.multiple) {
-					extraParam[serverParamName.label] = XAUtils
-							.enumLabelToValue(serverParamName.optionsArr, m
-									.get('value'));
-					;
-					$.extend(params, extraParam);
-				} else {
-					if (!_.isUndefined(serverParamName)) {
+				if (!_.isUndefined(serverParamName)) {
+					if (_.has(serverParamName, 'multiple')
+							&& serverParamName.multiple) {
+						extraParam[serverParamName.label] = XAUtils
+								.enumLabelToValue(serverParamName.optionsArr, m
+										.get('value'));
+						$.extend(params, extraParam);
+					} else {
 						extraParam[serverParamName.label] = m.get('value');
 						$.extend(params, extraParam);
 					}
diff --git a/security-admin/src/main/webapp/scripts/views/reports/AuditLayout.js b/security-admin/src/main/webapp/scripts/views/reports/AuditLayout.js
index 718a95d..9040c49 100644
--- a/security-admin/src/main/webapp/scripts/views/reports/AuditLayout.js
+++ b/security-admin/src/main/webapp/scripts/views/reports/AuditLayout.js
@@ -443,8 +443,8 @@ define(function(require) {
 			})
 			if(!_.isUndefined(App.sessionId)){
                                 App.vsHistory.admin = [] ;
-				query = '"Session Id": "'+App.sessionId+'"';
-                                App.vsHistory.admin.push(new Backbone.Model({'category':'Session Id', value:App.sessionId}));
+				query = '"Session ID": "'+App.sessionId+'"';
+                                App.vsHistory.admin.push(new Backbone.Model({'category':'Session ID', value:App.sessionId}));
 				delete App.sessionId;
                         }else{
                                 _.map(App.vsHistory.admin, function(a){ query += '"'+a.get('category')+'":"'+a.get('value')+'"'; });


[ranger] 13/39: RANGER-2258: Improve the policy list page to prompt users when the service is disabled

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit 282f2fdaf8c03626902d3f009052ba4f576fb170
Author: zhangqiang2 <zh...@zte.com.cn>
AuthorDate: Fri Oct 19 15:59:11 2018 +0800

    RANGER-2258: Improve the policy list page to prompt users when the service is disabled
    
    Signed-off-by: peng.jianhua <pe...@zte.com.cn>
---
 .../src/main/webapp/scripts/views/policies/RangerPolicyTableLayout.js | 4 ++--
 .../main/webapp/templates/policies/RangerPolicyTableLayout_tmpl.html  | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/security-admin/src/main/webapp/scripts/views/policies/RangerPolicyTableLayout.js b/security-admin/src/main/webapp/scripts/views/policies/RangerPolicyTableLayout.js
index 1150838..90ad83e 100644
--- a/security-admin/src/main/webapp/scripts/views/policies/RangerPolicyTableLayout.js
+++ b/security-admin/src/main/webapp/scripts/views/policies/RangerPolicyTableLayout.js
@@ -56,7 +56,7 @@ define(function(require){
 				rangerPolicyType : this.collection.queryParams['policyType'],
 				isRenderAccessTab : XAUtil.isRenderMasking(this.rangerServiceDefModel.get('dataMaskDef')) ? true 
                                         : XAUtil.isRenderRowFilter(this.rangerServiceDefModel.get('rowFilterDef')) ? true : false,
-                isNotAuditorAdminOrKmsAuditor : !(XAUtil.isAuditorOrKMSAuditor(SessionMgr))
+                isAddNewPolicyButtonShow : !(XAUtil.isAuditorOrKMSAuditor(SessionMgr)) && this.rangerService.get('isEnabled')
 			};
 		},
         
@@ -168,7 +168,7 @@ define(function(require){
 				gridOpts : {
 					row: Backgrid.Row.extend({}),
 					header : XABackgrid,
-					emptyText : 'No Policies found!'
+					emptyText : 'No Policies found!' + (this.rangerService.get('isEnabled') ? '' : ' The service is disabled!')
 				},
 			}));
 		},
diff --git a/security-admin/src/main/webapp/templates/policies/RangerPolicyTableLayout_tmpl.html b/security-admin/src/main/webapp/templates/policies/RangerPolicyTableLayout_tmpl.html
index bcd495c..a2b930f 100644
--- a/security-admin/src/main/webapp/templates/policies/RangerPolicyTableLayout_tmpl.html
+++ b/security-admin/src/main/webapp/templates/policies/RangerPolicyTableLayout_tmpl.html
@@ -45,7 +45,7 @@
             </div>
             <div class="span2">
                 <div class="clearfix btn-right">
-                    {{#if isNotAuditorAdminOrKmsAuditor}}
+                    {{#if isAddNewPolicyButtonShow}}
                             <a data-js="addNewPolicy" href="#!/service/{{rangerService.id}}/policies/create/{{this.rangerPolicyType}}" class="btn btn-primary " type="button">{{tt 'lbl.addNewPolicy'}} </a>
                     {{/if}}
                 </div>


[ranger] 20/39: RANGER-2277: Kylin repository config missing 'Common Name for Certificate'

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit 45392da42e911d7b3f59289e070caacfa4e378d6
Author: zhangqiang2 <zh...@zte.com.cn>
AuthorDate: Mon Nov 5 17:24:41 2018 +0800

    RANGER-2277: Kylin repository config missing 'Common Name for Certificate'
    
    Signed-off-by: peng.jianhua <pe...@zte.com.cn>
---
 .../main/resources/service-defs/ranger-servicedef-kylin.json  | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/agents-common/src/main/resources/service-defs/ranger-servicedef-kylin.json b/agents-common/src/main/resources/service-defs/ranger-servicedef-kylin.json
index 3e0f0a3..a6e76a0 100644
--- a/agents-common/src/main/resources/service-defs/ranger-servicedef-kylin.json
+++ b/agents-common/src/main/resources/service-defs/ranger-servicedef-kylin.json
@@ -88,6 +88,17 @@
 			"validationMessage": "",
 			"uiHint":"{\"TextFieldWithIcon\":true, \"info\": \"1.For one url, eg.<br>'http://&lt;ipaddr&gt;:7070'<br>2.For multiple urls (use , or ; delimiter), eg.<br>'http://&lt;ipaddr1&gt;:7070,http://&lt;ipaddr2&gt;:7070'\"}",
 			"label": "Kylin URL"
+		},
+
+		{
+			"itemId": 4,
+			"name": "commonNameForCertificate",
+			"type": "string",
+			"mandatory": false,
+			"validationRegEx":"",
+			"validationMessage": "",
+			"uiHint":"",
+			"label": "Common Name for Certificate"
 		}
 
 	],


[ranger] 36/39: RANGER-2306 : Add support for X-Forwarded-for header in Knox plugin

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit d57e363ad3b7d6d7927131460bcddf36529d3b54
Author: Vipin Rathor <v....@gmail.com>
AuthorDate: Thu Dec 6 15:46:01 2018 -0800

    RANGER-2306 : Add support for X-Forwarded-for header in Knox plugin
    
    Signed-off-by: Ramesh Mani <rm...@H12544.local>
---
 .../authorization/knox/KnoxRangerPlugin.java       | 13 +++++++++++
 .../authorization/knox/RangerPDPKnoxFilter.java    | 26 +++++++++++++++++++---
 2 files changed, 36 insertions(+), 3 deletions(-)

diff --git a/knox-agent/src/main/java/org/apache/ranger/authorization/knox/KnoxRangerPlugin.java b/knox-agent/src/main/java/org/apache/ranger/authorization/knox/KnoxRangerPlugin.java
index d248785..814aedd 100644
--- a/knox-agent/src/main/java/org/apache/ranger/authorization/knox/KnoxRangerPlugin.java
+++ b/knox-agent/src/main/java/org/apache/ranger/authorization/knox/KnoxRangerPlugin.java
@@ -19,6 +19,7 @@
 
 package org.apache.ranger.authorization.knox;
 
+import java.util.List;
 import java.util.Set;
 
 import org.apache.ranger.authorization.knox.KnoxRangerPlugin.KnoxConstants.AccessType;
@@ -56,6 +57,8 @@ public class KnoxRangerPlugin extends RangerBasePlugin {
 		Set<String> _groups;
 		String _clientIp;
 		String _clusterName;
+		String _remoteIp;
+		List<String> _forwardedAddresses;
 		
 		RequestBuilder service(String service) {
 			_service = service;
@@ -81,6 +84,14 @@ public class KnoxRangerPlugin extends RangerBasePlugin {
 			_clusterName = clusterName;
 			return this;
 		}
+		RequestBuilder remoteIp(String remoteIp) {
+			_remoteIp = remoteIp;
+			return this;
+		}
+		RequestBuilder forwardedAddresses(List<String> forwardedAddresses) {
+			_forwardedAddresses = forwardedAddresses;
+			return this;
+		}
 		void verifyBuildable() {
 			if (_topology == null) throw new IllegalStateException("_topology can't be null!");
 			if (_service == null) throw new IllegalStateException("_service can't be null!");
@@ -101,6 +112,8 @@ public class KnoxRangerPlugin extends RangerBasePlugin {
 			request.setUserGroups(_groups);
 			request.setResource(resource);
 			request.setClusterName(_clusterName);
+			request.setRemoteIPAddress(_remoteIp);
+			request.setForwardedAddresses(_forwardedAddresses);
 			return request;
 		}
 	}
diff --git a/knox-agent/src/main/java/org/apache/ranger/authorization/knox/RangerPDPKnoxFilter.java b/knox-agent/src/main/java/org/apache/ranger/authorization/knox/RangerPDPKnoxFilter.java
index f84a3e0..e75f314 100644
--- a/knox-agent/src/main/java/org/apache/ranger/authorization/knox/RangerPDPKnoxFilter.java
+++ b/knox-agent/src/main/java/org/apache/ranger/authorization/knox/RangerPDPKnoxFilter.java
@@ -21,7 +21,9 @@ package org.apache.ranger.authorization.knox;
 import java.io.IOException;
 import java.security.AccessController;
 import java.security.Principal;
+import java.util.Arrays;
 import java.util.HashSet;
+import java.util.List;
 import java.util.Set;
 
 import javax.security.auth.Subject;
@@ -31,6 +33,7 @@ import javax.servlet.FilterConfig;
 import javax.servlet.ServletException;
 import javax.servlet.ServletRequest;
 import javax.servlet.ServletResponse;
+import javax.servlet.http.HttpServletRequest;
 import javax.servlet.http.HttpServletResponse;
 
 import org.apache.commons.logging.Log;
@@ -40,6 +43,7 @@ import org.apache.knox.gateway.security.GroupPrincipal;
 import org.apache.knox.gateway.security.ImpersonatedPrincipal;
 import org.apache.knox.gateway.security.PrimaryPrincipal;
 import org.apache.ranger.audit.provider.MiscUtil;
+import org.apache.ranger.authorization.knox.KnoxRangerPlugin.RequestBuilder;
 import org.apache.ranger.plugin.policyengine.RangerAccessRequest;
 import org.apache.ranger.plugin.policyengine.RangerAccessResult;
 import org.apache.ranger.plugin.util.RangerPerfTracer;
@@ -131,21 +135,25 @@ public class RangerPDPKnoxFilter implements Filter {
 
 		String clientIp = request.getRemoteAddr();
 		String clusterName = plugin.getClusterName();
+		List<String> forwardedAddresses = getForwardedAddresses(request);
 
 		if (LOG.isDebugEnabled()) {
 			LOG.debug("Checking access primaryUser: " + primaryUser
 					+ ", impersonatedUser: " + impersonatedUser
 					+ ", effectiveUser: " + user + ", groups: " + groups
-					+ ", clientIp: " + clientIp + ", clusterName: "
-					+ clusterName);
+					+ ", clientIp: " + clientIp + ", clusterName: " + clusterName
+			    + ", remoteIp: " + clientIp + ", forwardedAddresses: " + forwardedAddresses);
 		}
-		RangerAccessRequest accessRequest = new KnoxRangerPlugin.RequestBuilder()
+
+		RangerAccessRequest accessRequest = new RequestBuilder()
 			.service(serviceName)
 			.topology(topologyName)
 			.user(user)
 			.groups(groups)
 			.clientIp(clientIp)
 			.clusterName(clusterName)
+			.remoteIp(clientIp)
+			.forwardedAddresses(forwardedAddresses)
 			.build();
 
 		boolean accessAllowed = false;
@@ -169,6 +177,18 @@ public class RangerPDPKnoxFilter implements Filter {
 		}
 	}
 
+	private List<String> getForwardedAddresses(ServletRequest request) {
+		List<String> forwardedAddresses = null;
+		if (request instanceof HttpServletRequest) {
+			HttpServletRequest httpRequest = (HttpServletRequest) request;
+			String xForwardedFor = httpRequest.getHeader("X-Forwarded-For");
+			if(xForwardedFor != null) {
+				forwardedAddresses = Arrays.asList(xForwardedFor.split(","));
+			}
+		}
+		return forwardedAddresses;
+	}
+
 	private void sendForbidden(HttpServletResponse res) {
 		sendErrorCode(res, 403);
 	}


[ranger] 34/39: RANGER-2294:Front-end and back-end email address regular expression should be the same

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit 4dc2fda26c135c259a62cc580040c8cc51966239
Author: zhangqiang2 <zh...@zte.com.cn>
AuthorDate: Wed Nov 28 16:10:05 2018 +0800

    RANGER-2294:Front-end and back-end email address regular expression should be the same
    
    Signed-off-by: peng.jianhua <pe...@zte.com.cn>
---
 .../src/main/webapp/scripts/modules/globalize/message/en.js         | 1 +
 security-admin/src/main/webapp/scripts/views/users/UserForm.js      | 6 ++++--
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/security-admin/src/main/webapp/scripts/modules/globalize/message/en.js b/security-admin/src/main/webapp/scripts/modules/globalize/message/en.js
index 2c0ee98..34e3387 100644
--- a/security-admin/src/main/webapp/scripts/modules/globalize/message/en.js
+++ b/security-admin/src/main/webapp/scripts/modules/globalize/message/en.js
@@ -469,6 +469,7 @@ define(function(require) {
                 lastNameValidationMsg  :'1. Last name should be start with alphabet / numeric / underscore / non-us characters.<br> 2. Allowed special character ._-@ and space. <br>3. Name length should be greater than one.',
                 setTimeZoneErrorMsg    :'Please select Start/End date for Time zone',
                 jsValidationMsg        :'1. JavaScript Condition Examples :\ncountry_code == \'USA\', time_range >= 900 && time_range <= 1800 etc.\n2. Dragging bottom-right corner of javascript condition editor(Textarea) can resizable',
+				emailAddressValidationMsg  :'1. Email address should be start with alphabet / numeric / underscore / non-us characters.<br> 2. Allowed special character <b>.-@</b> .<br>3. Email address length should be greater than 9 characters.<br> 4. Email address examples : abc@de.fg, A-C@D-.FG',
 			},
 			serverMsg : {
 
diff --git a/security-admin/src/main/webapp/scripts/views/users/UserForm.js b/security-admin/src/main/webapp/scripts/views/users/UserForm.js
index ee0d256..b11264f 100644
--- a/security-admin/src/main/webapp/scripts/views/users/UserForm.js
+++ b/security-admin/src/main/webapp/scripts/views/users/UserForm.js
@@ -105,9 +105,11 @@ define(function(require){
 					errorMsg    :localization.tt('validationMessages.lastNameValidationMsg'),
 				},
 				emailAddress : {
-					type		: 'Text',
+					type		: 'TextFieldWithIcon',
 					title		: localization.tt("lbl.emailAddress"),
-					validators  : ['email']
+					validators  : [{type:'regexp',regexp:/^[\w]([\-\.\w])+[\w]+@[\w]+[\w\-]+[\w]*\.([\w]+[\w\-]+[\w]*(\.[a-z][a-z|0-9]*)?)$/,
+						message :'Invalid email address'}],
+					errorMsg    :localization.tt('validationMessages.emailAddressValidationMsg'),
 				},
 				userRoleList : {
 					type : 'Select',


[ranger] 32/39: RANGER-2303:Add kylin-plugin infomation to README.txt

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit c5ba1ce0290a78c54667453b688d8ce3b2c659dc
Author: zhangqiang2 <zh...@zte.com.cn>
AuthorDate: Wed Dec 5 14:10:10 2018 +0800

    RANGER-2303:Add kylin-plugin infomation to README.txt
    
    Signed-off-by: peng.jianhua <pe...@zte.com.cn>
---
 README.txt | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/README.txt b/README.txt
index e54ce33..8aa1779 100644
--- a/README.txt
+++ b/README.txt
@@ -52,6 +52,7 @@ Build Process
    ranger-<version>-tagsync.tar.gz
    ranger-<version>-usersync.tar.gz
    ranger-<version>-yarn-plugin.tar.gz
+   ranger-<version>-kylin-plugin.tar.gz
 
 Importing Apache Ranger Project into Eclipse
 ============================================
@@ -77,6 +78,7 @@ Installation Host Information
     (f)  Kafka/Solr Plugin needs to be installed on their respective component hosts.
     (g)  YARN plugin needs to be installed on YARN Resource Manager hosts
     (h)  Sqoop plugin needs to be installed on Sqoop2 hosts
+    (i)  Kylin plugin needs to be installed on Kylin hosts
 
 Installation Process
 ~~~~~~~~~~~~~~~~~~~~


[ranger] 09/39: RANGER-2252:Permission Kafka Admin should not be part of Topic resource in Ranger Kafka resource definition

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit fbcdac076458afdaa6da09e87afed7ac2c1d1cc8
Author: rmani <rm...@hortonworks.com>
AuthorDate: Tue Oct 16 11:42:27 2018 -0700

    RANGER-2252:Permission Kafka Admin should not be part of Topic resource in Ranger Kafka resource definition
    
    Signed-off-by: rmani <rm...@hortonworks.com>
---
 .../src/main/resources/service-defs/ranger-servicedef-kafka.json        | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/agents-common/src/main/resources/service-defs/ranger-servicedef-kafka.json b/agents-common/src/main/resources/service-defs/ranger-servicedef-kafka.json
index 78ae9ea..800b123 100644
--- a/agents-common/src/main/resources/service-defs/ranger-servicedef-kafka.json
+++ b/agents-common/src/main/resources/service-defs/ranger-servicedef-kafka.json
@@ -24,7 +24,7 @@
 			"uiHint":"",
 			"label":"Topic",
 			"description":"Topic",
-			"accessTypeRestrictions": ["create", "delete", "configure", "alter_configs", "describe", "describe_configs", "consume", "publish", "kafka_admin"]
+			"accessTypeRestrictions": ["create", "delete", "configure", "alter_configs", "describe", "describe_configs", "consume", "publish"]
 		},
 		{
 			"itemId":2,


[ranger] 37/39: RANGER-2307: Better error message, and a NULL check for the native code

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit af6d18691ef0ad1c0fcdd4620ff14774ae790a58
Author: Zsombor Gegesy <zs...@apache.org>
AuthorDate: Fri Dec 7 10:47:53 2018 +0100

    RANGER-2307: Better error message, and a NULL check for the native code
---
 unixauthnative/src/main/c/credValidator.c | 7 ++++++-
 unixauthpam/src/main/c/pamCredValidator.c | 2 +-
 2 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/unixauthnative/src/main/c/credValidator.c b/unixauthnative/src/main/c/credValidator.c
index e426bdd..d79eb3b 100644
--- a/unixauthnative/src/main/c/credValidator.c
+++ b/unixauthnative/src/main/c/credValidator.c
@@ -22,6 +22,7 @@
 #include <string.h>
 #include <sys/types.h>
 #include <crypt.h>
+#include <errno.h>
 
 #define STRLEN 64
 
@@ -48,11 +49,15 @@ int main(int ac, char **av, char **ev)
 	spwd = getspnam(pwp->pw_name) ;
 
 	if (spwd == (struct spwd *)NULL) {
-		fprintf(stdout, "FAILED: unable to get (shadow) password for %s\n", username) ;
+		fprintf(stdout, "FAILED: unable to get (shadow) password for '%s', because '%s'\n", username, strerror(errno));
 		exit(1) ;
 	}
 	else {
 		char *gen = crypt(password,spwd->sp_pwdp) ;
+		if (gen == (char *)NULL) {
+			fprintf(stdout, "FAILED: crypt failed with: '%s'\n", strerror(errno));
+			exit(1);
+		}
 		if (strcmp(spwd->sp_pwdp,gen) == 0) {
 			fprintf(stdout, "OK:\n") ;
 			exit(0);
diff --git a/unixauthpam/src/main/c/pamCredValidator.c b/unixauthpam/src/main/c/pamCredValidator.c
index 60d38ae..8ec9f0e 100644
--- a/unixauthpam/src/main/c/pamCredValidator.c
+++ b/unixauthpam/src/main/c/pamCredValidator.c
@@ -83,7 +83,7 @@ int main(int ac, char **av, char **ev)
 
 	retval = pam_authenticate(pamh, 0);
 	if (retval != PAM_SUCCESS) {
-		fprintf(stdout, "FAILED: Password did not match.\n") ;
+		fprintf(stdout, "FAILED: Password did not match(%s).\n", pam_strerror(pamh, retval)) ;
 		if (pamh) {
 			pam_end(pamh, retval);
 		}		


[ranger] 14/39: RANGER-2264:Kafka default policies for new resources are not showing up in UI when upgrade is done from older version

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit ea8df629735f59e0385fd0184464c80e9ef8c694
Author: rmani <rm...@hortonworks.com>
AuthorDate: Tue Oct 23 18:20:01 2018 -0700

    RANGER-2264:Kafka default policies for new resources are not showing up in UI when upgrade is done from older version
    
    Signed-off-by: rmani <rm...@hortonworks.com>
---
 .../PatchForKafkaServiceDefUpdate_J10025.java      | 75 ++++++++++++++++++++--
 1 file changed, 71 insertions(+), 4 deletions(-)

diff --git a/security-admin/src/main/java/org/apache/ranger/patch/PatchForKafkaServiceDefUpdate_J10025.java b/security-admin/src/main/java/org/apache/ranger/patch/PatchForKafkaServiceDefUpdate_J10025.java
index 0ef1544..f15a278 100644
--- a/security-admin/src/main/java/org/apache/ranger/patch/PatchForKafkaServiceDefUpdate_J10025.java
+++ b/security-admin/src/main/java/org/apache/ranger/patch/PatchForKafkaServiceDefUpdate_J10025.java
@@ -19,6 +19,7 @@ package org.apache.ranger.patch;
 
 import org.apache.commons.lang.StringUtils;
 import org.apache.log4j.Logger;
+import org.apache.ranger.authorization.utils.JsonUtils;
 import org.apache.ranger.biz.RangerBizUtil;
 import org.apache.ranger.biz.ServiceDBStore;
 import org.apache.ranger.common.GUIDUtil;
@@ -53,6 +54,7 @@ import org.springframework.stereotype.Component;
 
 import java.util.ArrayList;
 import java.util.Arrays;
+import java.util.Date;
 import java.util.HashMap;
 import java.util.List;
 import java.util.Map;
@@ -292,9 +294,9 @@ public class PatchForKafkaServiceDefUpdate_J10025 extends BaseLoader {
 				xxPolicy.setGuid(guidUtil.genGUID());
 				xxPolicy.setAddedByUserId(currentUserId);
 				xxPolicy.setUpdatedByUserId(currentUserId);
-				RangerPolicy rangerPolicy = new RangerPolicy();
-				RangerPolicyResourceSignature resourceSignature = new RangerPolicyResourceSignature(rangerPolicy);
-				xxPolicy.setResourceSignature(resourceSignature.getSignature());
+				RangerPolicy rangerPolicy = getRangerPolicy(newResource,xxPortalUser,xxService);
+				xxPolicy.setPolicyText(JsonUtils.objectToJson(rangerPolicy));
+				xxPolicy.setResourceSignature(rangerPolicy.getResourceSignature());
 				XXPolicy createdPolicy = daoMgr.getXXPolicy().create(xxPolicy);
 
 				XXPolicyItem xxPolicyItem = new XXPolicyItem();
@@ -307,7 +309,7 @@ public class PatchForKafkaServiceDefUpdate_J10025 extends BaseLoader {
 				xxPolicyItem.setPolicyId(createdPolicy.getId());
 				XXPolicyItem createdXXPolicyItem = daoMgr.getXXPolicyItem().create(xxPolicyItem);
 
-				List<String> accessTypes = Arrays.asList("create", "delete", "configure", "alter_configs", "describe", "describe_configs", "consume", "publish", "kafka_admin","idempotent_write");
+				List<String> accessTypes = getAccessTypes();
 				for (int i = 0; i < accessTypes.size(); i++) {
 					XXAccessTypeDef xAccTypeDef = daoMgr.getXXAccessTypeDef().findByNameAndServiceId(accessTypes.get(i),
 							xxPolicy.getService());
@@ -378,4 +380,69 @@ public class PatchForKafkaServiceDefUpdate_J10025 extends BaseLoader {
 		}
 		logger.info("<== createDefaultPolicyForNewResources ");
 	}
+
+
+	private RangerPolicy getRangerPolicy(String newResource, XXPortalUser xxPortalUser, XXService xxService) {
+		RangerPolicy policy = new RangerPolicy();
+
+		List<RangerPolicy.RangerPolicyItemAccess> accesses = getPolicyItemAccesses();
+		List<String> users = new ArrayList<>(DEFAULT_POLICY_USERS);
+		List<String> groups = new ArrayList<>();
+		List<RangerPolicy.RangerPolicyItemCondition> conditions = new ArrayList<>();
+		List<RangerPolicy.RangerPolicyItem> policyItems = new ArrayList<>();
+		RangerPolicy.RangerPolicyItem rangerPolicyItem = new RangerPolicy.RangerPolicyItem();
+		rangerPolicyItem.setAccesses(accesses);
+		rangerPolicyItem.setConditions(conditions);
+		rangerPolicyItem.setGroups(groups);
+		rangerPolicyItem.setUsers(users);
+		rangerPolicyItem.setDelegateAdmin(false);
+
+		policyItems.add(rangerPolicyItem);
+
+		Map<String, RangerPolicy.RangerPolicyResource> policyResource = new HashMap<>();
+		RangerPolicy.RangerPolicyResource rangerPolicyResource = new RangerPolicy.RangerPolicyResource();
+		rangerPolicyResource.setIsExcludes(false);
+		rangerPolicyResource.setIsRecursive(false);
+		rangerPolicyResource.setValue("*");
+		String policyResourceName = KAFKA_RESOURCE_CLUSTER;
+		if ("all - delegationtoken".equals(newResource)) {
+			policyResourceName = KAFKA_RESOURCE_DELEGATIONTOKEN;
+		}
+		policyResource.put(policyResourceName, rangerPolicyResource);
+		policy.setCreateTime(new Date());
+		policy.setDescription(newResource);
+		policy.setIsEnabled(true);
+		policy.setName(newResource);
+		policy.setCreatedBy(xxPortalUser.getLoginId());
+		policy.setUpdatedBy(xxPortalUser.getLoginId());
+		policy.setUpdateTime(new Date());
+		policy.setService(xxService.getName());
+		policy.setIsAuditEnabled(true);
+		policy.setPolicyItems(policyItems);
+		policy.setResources(policyResource);
+		policy.setPolicyType(0);
+		policy.setId(0L);
+		policy.setGuid("");
+		policy.setPolicyLabels(new ArrayList<>());
+		policy.setVersion(1L);
+		RangerPolicyResourceSignature resourceSignature = new RangerPolicyResourceSignature(policy);
+		policy.setResourceSignature(resourceSignature.getSignature());
+		return policy;
+	}
+
+	private List<String> getAccessTypes() {
+		List<String> accessTypes = Arrays.asList("create", "delete", "configure", "alter_configs", "describe", "describe_configs", "consume", "publish", "kafka_admin", "idempotent_write");
+		return accessTypes;
+	}
+
+	private ArrayList<RangerPolicy.RangerPolicyItemAccess> getPolicyItemAccesses() {
+		ArrayList<RangerPolicy.RangerPolicyItemAccess> rangerPolicyItemAccesses = new ArrayList<>();
+		for(String type:getAccessTypes()) {
+			RangerPolicy.RangerPolicyItemAccess policyItemAccess = new  RangerPolicy.RangerPolicyItemAccess();
+			policyItemAccess.setType(type);
+			policyItemAccess.setIsAllowed(true);
+			rangerPolicyItemAccesses.add(policyItemAccess);
+		}
+		return rangerPolicyItemAccesses;
+	}
 }
\ No newline at end of file


[ranger] 12/39: RANGER-2263: Removed unnecessary explicit dependency for apache commons compress jar in Ranger

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit dccd0dcc757b43380cf2ea88937f30faff7b00dd
Author: Sailaja Polavarapu <sp...@hortonworks.com>
AuthorDate: Tue Oct 23 09:50:45 2018 -0700

    RANGER-2263: Removed unnecessary explicit dependency for apache commons compress jar in Ranger
---
 pom.xml                       |  2 --
 security-admin/pom.xml        | 12 ++++++------
 src/main/assembly/tagsync.xml |  2 +-
 3 files changed, 7 insertions(+), 9 deletions(-)

diff --git a/pom.xml b/pom.xml
index f3b1c8e..3b5df69 100644
--- a/pom.xml
+++ b/pom.xml
@@ -86,7 +86,6 @@
         <atlas.gson.version>2.5</atlas.gson.version>
         <atlas.jackson.version>2.9.2</atlas.jackson.version>
         <atlas.jettison.version>1.3.7</atlas.jettison.version>
-        <atlas.commons.compress.version>1.4.1</atlas.commons.compress.version>
         <atlas.commons.logging.version>1.1.3</atlas.commons.logging.version>
         <bouncycastle.version>1.55</bouncycastle.version>
         <c3p0.version>0.9.5.2</c3p0.version>
@@ -96,7 +95,6 @@
         <commons.cli.version>1.2</commons.cli.version>
         <commons.codec.version>1.9</commons.codec.version>
         <commons.collections.version>3.2.2</commons.collections.version>
-        <commons.compress.version>1.8.1</commons.compress.version>
         <commons.configuration.version>1.10</commons.configuration.version>
         <commons.dbcp.version>1.4</commons.dbcp.version>
         <commons.digester.version>2.1</commons.digester.version>
diff --git a/security-admin/pom.xml b/security-admin/pom.xml
index 243e430..1e816ff 100644
--- a/security-admin/pom.xml
+++ b/security-admin/pom.xml
@@ -83,11 +83,6 @@
             <version>${commons.collections.version}</version>
         </dependency>
         <dependency>
-            <groupId>org.apache.commons</groupId>
-            <artifactId>commons-compress</artifactId>
-            <version>${commons.compress.version}</version>
-        </dependency>
-        <dependency>
             <groupId>commons-configuration</groupId>
             <artifactId>commons-configuration</artifactId>
             <version>${commons.configuration.version}</version>
@@ -368,6 +363,10 @@
             <version>${hadoop.version}</version>
             <exclusions>
 		<exclusion>
+		    <groupId>org.apache.commons</groupId>
+            	    <artifactId>commons-compress</artifactId>
+                </exclusion>
+		<exclusion>
                     <groupId>commons-httpclient</groupId>
                     <artifactId>commons-httpclient</artifactId>
                 </exclusion>
@@ -542,7 +541,8 @@
 								WEB-INF/lib/spring-*.SEC03.jar,
 								WEB-INF/lib/spring-*.RC3.jar,
 								WEB-INF/lib/spring-2.*.jar,
-								WEB-INF/lib/jetty-*.jar
+								WEB-INF/lib/jetty-*.jar,
+								WEB-INF/lib/commons-compress-*.jar
 							</packagingExcludes>
                             <warSourceDirectory>${project.build.directory}/${project.build.finalName}</warSourceDirectory>
                         </configuration>
diff --git a/src/main/assembly/tagsync.xml b/src/main/assembly/tagsync.xml
index d1b83df..0b6596f 100644
--- a/src/main/assembly/tagsync.xml
+++ b/src/main/assembly/tagsync.xml
@@ -45,9 +45,9 @@
 					<include>org.apache.atlas:atlas-client-v2:jar:${atlas.version}</include>
 					<include>org.apache.atlas:atlas-client-common:jar:${atlas.version}</include>
 					<include>org.apache.atlas:atlas-common:jar:${atlas.version}</include>
-					<include>org.apache.commons:commons-compress:jar:${atlas.commons.compress.version}</include>
 					<include>org.apache.hadoop:hadoop-auth</include>
 					<include>org.apache.hadoop:hadoop-common</include>
+					<include>org.apache.commons:commons-compress</include>
 					<include>org.apache.kafka:kafka_${scala.binary.version}:jar:${kafka.version}</include>
 					<include>org.apache.kafka:kafka-clients:jar:${kafka.version}</include>
 					<include>org.apache.ranger:credentialbuilder</include>


[ranger] 25/39: RANGER-2284: Unable to build image using docker

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit 187d8e8f9bebf989244e37f8100e72e4cfd6e7df
Author: Don Bosco Durai <bo...@apache.org>
AuthorDate: Thu Nov 15 22:26:20 2018 -0800

    RANGER-2284: Unable to build image using docker
---
 build_ranger_using_docker.sh | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/build_ranger_using_docker.sh b/build_ranger_using_docker.sh
index 82b7179..fa77d91 100755
--- a/build_ranger_using_docker.sh
+++ b/build_ranger_using_docker.sh
@@ -22,12 +22,12 @@
 #5. To delete the image, run "[sudo] docker rmi ranger_dev"
 
 #Usage: [sudo] ./build_ranger_using_docker.sh [-build_image] mvn  <build params>
-#Example 1 (default no param): (mvn -DskipTests=true clean compile package install assembly:assembly)
-#Example 2 (Regular build): ./build_ranger_using_docker.sh mvn clean install -DskipTests=true 
-#Example 3 (Recreate Docker image): ./build_ranger_using_docker.sh mvn -build_image clean install -DskipTests=true 
+#Example 1 (default no param): (mvn -Pall -DskipTests=true clean compile package install assembly:assembly)
+#Example 2 (Regular build): ./build_ranger_using_docker.sh mvn -Pall clean install -DskipTests=true
+#Example 3 (Recreate Docker image): ./build_ranger_using_docker.sh mvn -Pall -build_image clean install -DskipTests=true 
 #Notes: To remove build image manually, run "docker rmi ranger_dev" or "sudo docker rmi ranger_dev"
 
-default_command="mvn -DskipTests=true clean compile package install assembly:assembly"
+default_command="mvn -Pall -DskipTests=true clean compile package install assembly:assembly"
 build_image=0
 if [ "$1" = "-build_image" ]; then
     build_image=1
@@ -75,14 +75,14 @@ ENV JAVA_HOME /usr/java/latest
 ENV  PATH $JAVA_HOME/bin:$PATH
 
 
-ADD https://www.apache.org/dist/maven/maven-3/3.5.3/binaries/apache-maven-3.5.3-bin.tar.gz.sha1 /tools
-ADD http://www-us.apache.org/dist/maven/maven-3/3.5.3/binaries/apache-maven-3.5.3-bin.tar.gz /tools
-RUN sha1sum  apache-maven-3.5.3-bin.tar.gz | cut -f 1 -d " " > tmp.sha1
+ADD https://www.apache.org/dist/maven/maven-3/3.5.4/binaries/apache-maven-3.5.4-bin.tar.gz.sha512 /tools
+ADD http://www-us.apache.org/dist/maven/maven-3/3.5.4/binaries/apache-maven-3.5.4-bin.tar.gz /tools
+RUN sha512sum  apache-maven-3.5.4-bin.tar.gz | cut -f 1 -d " " > tmp.sha1
 
-RUN diff -w tmp.sha1 apache-maven-3.5.3-bin.tar.gz.sha1
+RUN diff -w tmp.sha1 apache-maven-3.5.4-bin.tar.gz.sha512
 
-RUN tar xfz apache-maven-3.5.3-bin.tar.gz
-RUN ln -sf /tools/apache-maven-3.5.3 /tools/maven
+RUN tar xfz apache-maven-3.5.4-bin.tar.gz
+RUN ln -sf /tools/apache-maven-3.5.4 /tools/maven
 
 ENV  PATH /tools/maven/bin:$PATH
 ENV MAVEN_OPTS "-Xmx2048m -XX:MaxPermSize=512m"


[ranger] 04/39: RANGER-2228: Updated docs for Apache Ranger 1.2.0 release

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit 4d12157404789685cf740b34db209f964c02df64
Author: Velmurugan Periasamy <ve...@apache.org>
AuthorDate: Thu Oct 4 13:50:39 2018 -0400

    RANGER-2228: Updated docs for Apache Ranger 1.2.0 release
    
    Signed-off-by: Velmurugan Periasamy <ve...@apache.org>
---
 docs/pom.xml                    | 108 ++++++++++++++++++++++++++++++++++++++++
 docs/src/site/site.xml          |   1 +
 docs/src/site/xdoc/download.xml |  12 ++++-
 3 files changed, 120 insertions(+), 1 deletion(-)

diff --git a/docs/pom.xml b/docs/pom.xml
index 9c6426d..f14867c 100644
--- a/docs/pom.xml
+++ b/docs/pom.xml
@@ -414,10 +414,22 @@
     </developers>
     <contributors>
         <contributor>
+          	<name>Alejandro Fernandez</name>
+          	<organization></organization>
+        </contributor>
+        <contributor>
           	<name>Aneela Saleem</name>
           	<organization></organization>
         </contributor>
         <contributor>
+          	<name>Ankit Singhal</name>
+          	<organization></organization>
+        </contributor>
+        <contributor>
+          	<name>Anna Shaverdian</name>
+          	<organization></organization>
+        </contributor>
+        <contributor>
           	<name>Arshad Mohammad</name>
           	<organization>Huawei</organization>
         </contributor>
@@ -430,6 +442,10 @@
           	<organization></organization>
         </contributor>
         <contributor>
+          	<name>Bhavik Patel</name>
+          	<organization>Freestone Infotech</organization>
+        </contributor>
+        <contributor>
           	<name>Bolke de Bruin</name>
           	<organization>ING</organization>
         </contributor>
@@ -438,10 +454,30 @@
           	<organization></organization>
         </contributor>
         <contributor>
+          	<name>Dhaval Rajpara</name>
+          	<organization></organization>
+        </contributor>
+        <contributor>
+          	<name>Dongying Jiao</name>
+          	<organization></organization>
+        </contributor>
+        <contributor>
           	<name>Edward Zhang</name>
           	<organization></organization>
         </contributor>
         <contributor>
+          	<name>Endre Kovacs</name>
+          	<organization></organization>
+        </contributor>
+        <contributor>
+          	<name>Fatima Amjad Khan</name>
+          	<organization>Freestone Infotech</organization>
+        </contributor>
+        <contributor>
+          	<name>Haihui Xu</name>
+          	<organization></organization>
+        </contributor>
+        <contributor>
           	<name>Hanish Bansal</name>
           	<organization></organization>
         </contributor>
@@ -450,10 +486,26 @@
           	<organization></organization>
         </contributor>
         <contributor>
+          	<name>Kent Yao</name>
+          	<organization></organization>
+        </contributor>
+        <contributor>
+          	<name>Kevin Risden</name>
+          	<organization></organization>
+        </contributor>
+        <contributor>
+          	<name>Koji Kawamura</name>
+          	<organization></organization>
+        </contributor>
+        <contributor>
           	<name>Mack Hendricks</name>
           	<organization>Hortonworks Inc.,</organization>
         </contributor>
         <contributor>
+          	<name>Madhavi Amirneni</name>
+          	<organization></organization>
+        </contributor>
+        <contributor>
           	<name>Mani Raman</name>
           	<organization></organization>
         </contributor>
@@ -462,14 +514,54 @@
           	<organization>Freestone Infotech</organization>
         </contributor>
         <contributor>
+          	<name>Nicholas Hughes</name>
+          	<organization></organization>
+        </contributor>
+        <contributor>
+          	<name>Nigel Jones</name>
+          	<organization></organization>
+        </contributor>
+        <contributor>
+          	<name>Nikhil Purbhe</name>
+          	<organization>Freestone Infotech</organization>
+        </contributor>
+        <contributor>
+          	<name>Nitin Galave</name>
+          	<organization>Freestone Infotech</organization>
+        </contributor>
+        <contributor>
+          	<name>Nixon Rodrigues</name>
+          	<organization></organization>
+        </contributor>
+        <contributor>
+          	<name>Paul Otto</name>
+          	<organization></organization>
+        </contributor>
+        <contributor>
+          	<name>Peng Xing</name>
+          	<organization></organization>
+        </contributor>
+        <contributor>
           	<name>Rich Haase</name>
           	<organization>Pandora</organization>
         </contributor>
         <contributor>
+          	<name>Rohit Sinha</name>
+          	<organization></organization>
+        </contributor>
+        <contributor>
+          	<name>Shi Wang</name>
+          	<organization></organization>
+        </contributor>
+        <contributor>
           	<name>Sree Vaddi</name>
           	<organization></organization>
         </contributor>
         <contributor>
+          	<name>Suneel Marthi</name>
+          	<organization></organization>
+        </contributor>
+        <contributor>
           	<name>Tushar Dudhatra</name>
           	<organization></organization>
         </contributor>
@@ -477,6 +569,22 @@
           	<name>Varun Rao</name>
           	<organization>Accenture</organization>
         </contributor>
+        <contributor>
+          	<name>Vishal Suvagia</name>
+          	<organization>Freestone Infotech</organization>
+        </contributor>
+        <contributor>
+          	<name>Wang Yuan</name>
+          	<organization></organization>
+        </contributor>
+        <contributor>
+          	<name>Yan Zhou</name>
+          	<organization></organization>
+        </contributor>
+        <contributor>
+          	<name>Yujie Li</name>
+          	<organization></organization>
+        </contributor>
     </contributors>
     <organization>
         <name>Apache Software Foundation</name>
diff --git a/docs/src/site/site.xml b/docs/src/site/site.xml
index 3da00cb..19c7bee 100644
--- a/docs/src/site/site.xml
+++ b/docs/src/site/site.xml
@@ -62,6 +62,7 @@ under the License.
 	    <item name="Security Advisories" href="https://cwiki.apache.org/confluence/display/RANGER/Vulnerabilities+found+in+Ranger" />
     </menu>
     <menu name="Releases">
+        <item name="1.2.0" href="https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+1.2.0+-+Release+Notes" />
         <item name="1.1.0" href="https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+1.1.0+-+Release+Notes" />
         <item name="1.0.0" href="https://cwiki.apache.org/confluence/display/RANGER/1.0.0+Release+-+Apache+Ranger" />
         <item name="0.7.1" href="https://cwiki.apache.org/confluence/display/RANGER/0.7.1+Release+-+Apache+Ranger" />
diff --git a/docs/src/site/xdoc/download.xml b/docs/src/site/xdoc/download.xml
index 91f7cf1..1b672ec 100644
--- a/docs/src/site/xdoc/download.xml
+++ b/docs/src/site/xdoc/download.xml
@@ -31,7 +31,17 @@ LICENSE.txt and NOTICE.txt files contained in each release artifact.
 <ul>
 <li>
 <p>
-Current Stable release is Apache Ranger 1.1.0:
+Current Stable release is Apache Ranger 1.2.0:
+</p>
+<p>
+<a href="https://www.apache.org/dyn/closer.lua/ranger/1.2.0/apache-ranger-1.2.0.tar.gz">apache-ranger-1.2.0.tar.gz</a>
+(<a href="https://www.apache.org/dist/ranger/1.2.0/apache-ranger-1.2.0.tar.gz.asc">PGP</a>)
+(<a href="https://www.apache.org/dist/ranger/1.2.0/apache-ranger-1.2.0.tar.gz.mds">Digests</a>)
+</p>
+</li>
+<li>
+<p>
+An older branch release is Apache Ranger 1.1.0:
 </p>
 <p>
 <a href="https://www.apache.org/dyn/closer.lua/ranger/1.1.0/apache-ranger-1.1.0.tar.gz">apache-ranger-1.1.0.tar.gz</a>


[ranger] 15/39: RANGER-2257:Add policyID to error message when click the Access log of Audit

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit 41e0b90c1dc4d3552c3d44a9afb93f745e07fc49
Author: zhangqiang2 <zh...@zte.com.cn>
AuthorDate: Fri Oct 19 11:21:45 2018 +0800

    RANGER-2257:Add policyID to error message when click the Access log of Audit
    
    Signed-off-by: peng.jianhua <pe...@zte.com.cn>
---
 security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java b/security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java
index 63c9432..f2d61d3 100644
--- a/security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java
+++ b/security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java
@@ -2838,7 +2838,7 @@ public class ServiceDBStore extends AbstractServiceStore {
 				AppConstants.CLASS_TYPE_RANGER_POLICY, policyId);
 
 		if (xDataHist == null) {
-			String errMsg = "No policy history found for given time: " + eventTime;
+			String errMsg = "No policy history found for given policy ID: " + policyId + " and event time: " + eventTime;
 			LOG.error(errMsg);
 			throw restErrorUtil.createRESTException(errMsg, MessageEnums.DATA_NOT_FOUND);
 		}


[ranger] 17/39: RANGER-2265: Added all ranger modules to linux profile so that all are built by default for unix environments

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit ec0c3b40199a95c6d9059d6b18f7c2f5b2f6f35a
Author: Sailaja Polavarapu <sp...@hortonworks.com>
AuthorDate: Wed Oct 31 11:00:38 2018 -0700

    RANGER-2265: Added all ranger modules to linux profile so that all are built by default for unix environments
---
 pom.xml | 74 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 74 insertions(+)

diff --git a/pom.xml b/pom.xml
index 3b5df69..c7895c5 100644
--- a/pom.xml
+++ b/pom.xml
@@ -585,8 +585,82 @@
                 </os>
             </activation>
             <modules>
+		<module>jisql</module>
+                <module>agents-audit</module>
+                <module>agents-common</module>
+                <module>agents-cred</module>
+                <module>agents-installer</module>
+                <module>credentialbuilder</module>
+                <module>embeddedwebserver</module>
+                <module>kms</module>
+                <module>hbase-agent</module>
+                <module>hdfs-agent</module>
+                <module>hive-agent</module>
+                <module>knox-agent</module>
+                <module>storm-agent</module>
+                <module>plugin-yarn</module>
+                <module>security-admin</module>
+                <module>plugin-kafka</module>
+                <module>plugin-solr</module>
+                <module>plugin-nifi</module>
+                <module>plugin-nifi-registry</module>
+                <module>ugsync</module>
+                <module>ugsync/ldapconfigchecktool/ldapconfigcheck</module>
+                <module>unixauthclient</module>
+                <module>unixauthservice</module>
+                <module>ranger-util</module>
+                <module>plugin-kms</module>
+                <module>tagsync</module>
+                <module>ranger-hdfs-plugin-shim</module>
+                <module>ranger-plugin-classloader</module>
+                <module>ranger-hive-plugin-shim</module>
+                <module>ranger-hbase-plugin-shim</module>
+                <module>ranger-knox-plugin-shim</module>
+                <module>ranger-yarn-plugin-shim</module>
+                <module>ranger-storm-plugin-shim</module>
+                <module>ranger-kafka-plugin-shim</module>
+                <module>ranger-solr-plugin-shim</module>
+                <module>ranger-atlas-plugin-shim</module>
+                <module>ranger-kms-plugin-shim</module>
+                <module>ranger-examples</module>
+                <module>ranger-tools</module>
+                <module>plugin-atlas</module>
+                <module>plugin-sqoop</module>
+                <module>ranger-sqoop-plugin-shim</module>
+                <module>plugin-kylin</module>
+                <module>ranger-kylin-plugin-shim</module>
                 <module>unixauthnative</module>
             </modules>
+            <build>
+                <plugins>
+                    <plugin>
+                        <artifactId>maven-assembly-plugin</artifactId>
+                        <version>2.2-beta-5</version>
+			<configuration>
+                            <descriptors>
+                                <descriptor>src/main/assembly/hdfs-agent.xml</descriptor>
+                                <descriptor>src/main/assembly/hive-agent.xml</descriptor>
+                                <descriptor>src/main/assembly/hbase-agent.xml</descriptor>
+                                <descriptor>src/main/assembly/knox-agent.xml</descriptor>
+                                <descriptor>src/main/assembly/storm-agent.xml</descriptor>
+                                <descriptor>src/main/assembly/plugin-kafka.xml</descriptor>
+                                <descriptor>src/main/assembly/plugin-yarn.xml</descriptor>
+                                <descriptor>src/main/assembly/plugin-solr.xml</descriptor>
+                                <descriptor>src/main/assembly/admin-web.xml</descriptor>
+                                <descriptor>src/main/assembly/usersync.xml</descriptor>
+                                <descriptor>src/main/assembly/tagsync.xml</descriptor>
+                                <descriptor>src/main/assembly/migration-util.xml</descriptor>
+                                <descriptor>src/main/assembly/kms.xml</descriptor>
+                                <descriptor>src/main/assembly/ranger-tools.xml</descriptor>
+                                <descriptor>src/main/assembly/ranger-src.xml</descriptor>
+                                <descriptor>src/main/assembly/plugin-atlas.xml</descriptor>
+                                <descriptor>src/main/assembly/plugin-sqoop.xml</descriptor>
+                                <descriptor>src/main/assembly/plugin-kylin.xml</descriptor>
+                            </descriptors>
+                        </configuration>
+                    </plugin>
+                </plugins>
+            </build>
         </profile>
         <profile>
             <id>linux-pam</id>


[ranger] 23/39: RANGER-2049: Fixed an issue where doAs User role is not set properly

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit b9f698697b2d5e0bae86593bc8e8eb1c4190b2f2
Author: Sailaja Polavarapu <sp...@hortonworks.com>
AuthorDate: Fri Nov 16 13:39:30 2018 -0800

    RANGER-2049: Fixed an issue where doAs User role is not set properly
---
 .../security/web/filter/RangerKRBAuthenticationFilter.java     | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerKRBAuthenticationFilter.java b/security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerKRBAuthenticationFilter.java
index 178f31e..5c825d8 100644
--- a/security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerKRBAuthenticationFilter.java
+++ b/security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerKRBAuthenticationFilter.java
@@ -270,16 +270,16 @@ public class RangerKRBAuthenticationFilter extends RangerKrbFilter {
 							final List<GrantedAuthority> grantedAuths = new ArrayList<>();
 							grantedAuths.add(new SimpleGrantedAuthority(rangerLdapDefaultRole));
 							final UserDetails principal = new User(doAsUser, "", grantedAuths);
-							final Authentication finalAuthentication = new UsernamePasswordAuthenticationToken(principal, "", grantedAuths);
+							Authentication authentication = new UsernamePasswordAuthenticationToken(principal, "", grantedAuths);
 							WebAuthenticationDetails webDetails = new WebAuthenticationDetails(request);
-							((AbstractAuthenticationToken) finalAuthentication).setDetails(webDetails);
-							SecurityContextHolder.getContext().setAuthentication(finalAuthentication);
+							((AbstractAuthenticationToken) authentication).setDetails(webDetails);
+							authentication = getGrantedAuthority(authentication);
+							SecurityContextHolder.getContext().setAuthentication(authentication);
 							request.setAttribute("spnegoEnabled", true);
+							LOG.info("Logged into Ranger as doAsUser = " + doAsUser + ", by authenticatedUser=" + authToken.getUserName());
 						}
 
 					}
-					LOG.info("Logged into Ranger as doAsUser = " + doAsUser + ", by authenticatedUser=" + authToken.getUserName());
-
 
 				}else {
 					//if we get the userName from the token then log into ranger using the same user


[ranger] 29/39: RANGER-2244: Tomcat Security Vulnerability Alert. The version of the tomcat for ranger should upgrade to 7.0.91 or later.

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit 7cbfd8bb88bd9b557bac46ad1a2c67ee08dbcdc9
Author: zhangqiang2 <zh...@zte.com.cn>
AuthorDate: Thu Nov 15 16:47:04 2018 +0800

    RANGER-2244: Tomcat Security Vulnerability Alert. The version of the tomcat for ranger should upgrade to 7.0.91 or later.
    
    Signed-off-by: Mehul Parikh <me...@apache.org>
---
 .../src/main/java/org/apache/ranger/server/tomcat/EmbeddedServer.java | 4 ----
 pom.xml                                                               | 2 +-
 2 files changed, 1 insertion(+), 5 deletions(-)

diff --git a/embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/EmbeddedServer.java b/embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/EmbeddedServer.java
index eac0dac..8d32352 100644
--- a/embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/EmbeddedServer.java
+++ b/embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/EmbeddedServer.java
@@ -27,7 +27,6 @@ import java.util.Iterator;
 import java.util.Properties;
 import java.util.logging.Logger;
 import java.util.List;
-import javax.servlet.ServletException;
 
 import org.apache.catalina.Context;
 import org.apache.catalina.LifecycleException;
@@ -222,9 +221,6 @@ public class EmbeddedServer {
 			webappCtx.init();
 			LOG.info("Finished init of webapp [" + webContextName
 					+ "] = path [" + webapp_dir + "].");
-		} catch (ServletException e1) {
-			LOG.severe("Tomcat Server failed to add webapp:" + e1.toString());
-			e1.printStackTrace();
 		} catch (LifecycleException lce) {
 			LOG.severe("Tomcat Server failed to start webapp:" + lce.toString());
 			lce.printStackTrace();
diff --git a/pom.xml b/pom.xml
index c7895c5..da44d61 100644
--- a/pom.xml
+++ b/pom.xml
@@ -166,7 +166,7 @@
         <sqoop.version>1.99.7</sqoop.version>
         <storm.version>1.2.0</storm.version>
         <sun-jersey-bundle.version>1.19</sun-jersey-bundle.version>
-        <tomcat.embed.version>7.0.90</tomcat.embed.version>
+        <tomcat.embed.version>7.0.91</tomcat.embed.version>
         <velocity.version>1.7</velocity.version>
         <zookeeper.version>3.4.6</zookeeper.version>
     </properties>


[ranger] 08/39: RANGER-2237: Upgrade Kylin version to 2.5.0

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit f5f7f33585930762463409eb69f615b5143aaf2a
Author: zhangqiang2 <zh...@zte.com.cn>
AuthorDate: Mon Oct 8 09:51:12 2018 +0800

    RANGER-2237: Upgrade Kylin version to 2.5.0
    
    Signed-off-by: peng.jianhua <pe...@zte.com.cn>
---
 pom.xml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/pom.xml b/pom.xml
index 308ec1f..5f17305 100644
--- a/pom.xml
+++ b/pom.xml
@@ -186,7 +186,7 @@
         <kafka.version>2.0.0</kafka.version>
         <kerby.version>1.0.0</kerby.version>
         <knox.gateway.version>1.1.0</knox.gateway.version>
-        <kylin.version>2.3.0</kylin.version>
+        <kylin.version>2.5.0</kylin.version>
         <libpam4j.version>1.10</libpam4j.version>
         <local.lib.dir>${project.basedir}/../lib/local</local.lib.dir>
         <log4j.version>1.2.17</log4j.version>


[ranger] 02/39: RANGER-2231 - Upgrade to Knox 1.1.0

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit b3c96004e045364ab33ac7b64f5c1a7fb4feaa27
Author: Colm O hEigeartaigh <co...@apache.org>
AuthorDate: Wed Sep 26 19:08:47 2018 -0400

    RANGER-2231 - Upgrade to Knox 1.1.0
    
    Signed-off-by: Colm O hEigeartaigh <co...@apache.org>
---
 pom.xml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/pom.xml b/pom.xml
index 29d3740..be3d05b 100644
--- a/pom.xml
+++ b/pom.xml
@@ -185,7 +185,7 @@
         <junit.version>4.12</junit.version>
         <kafka.version>2.0.0</kafka.version>
         <kerby.version>1.0.0</kerby.version>
-        <knox.gateway.version>1.0.0</knox.gateway.version>
+        <knox.gateway.version>1.1.0</knox.gateway.version>
         <kylin.version>2.3.0</kylin.version>
         <libpam4j.version>1.10</libpam4j.version>
         <local.lib.dir>${project.basedir}/../lib/local</local.lib.dir>


[ranger] 16/39: RANGER-2248: Sorting does not work in AbstractPredicateUtil.java

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit 156f48f5aded7fbc1aabba33151ddbb101bb9b01
Author: zhangqiang2 <zh...@zte.com.cn>
AuthorDate: Thu Oct 11 15:26:01 2018 +0800

    RANGER-2248: Sorting does not work in AbstractPredicateUtil.java
    
    Signed-off-by: peng.jianhua <pe...@zte.com.cn>
---
 .../main/java/org/apache/ranger/plugin/store/AbstractPredicateUtil.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/agents-common/src/main/java/org/apache/ranger/plugin/store/AbstractPredicateUtil.java b/agents-common/src/main/java/org/apache/ranger/plugin/store/AbstractPredicateUtil.java
index 4d2bc62..7446df6 100644
--- a/agents-common/src/main/java/org/apache/ranger/plugin/store/AbstractPredicateUtil.java
+++ b/agents-common/src/main/java/org/apache/ranger/plugin/store/AbstractPredicateUtil.java
@@ -101,7 +101,7 @@ public class AbstractPredicateUtil {
 	}
 
 	public Comparator<RangerBaseModelObject> getSorter(SearchFilter filter) {
-		String sortBy = filter == null ? null : filter.getParam(SearchFilter.SORT_BY);
+		String sortBy = filter == null ? null : filter.getSortBy();
 
 		if(StringUtils.isEmpty(sortBy)) {
 			return null;


[ranger] 03/39: RANGER-2239 - Update to surefire 2.21.0

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit c6fe2310694f5862ea0886657118dc43ab6a0746
Author: Colm O hEigeartaigh <co...@apache.org>
AuthorDate: Tue Oct 2 09:50:27 2018 +0100

    RANGER-2239 - Update to surefire 2.21.0
    
    Signed-off-by: Colm O hEigeartaigh <co...@apache.org>
---
 pom.xml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/pom.xml b/pom.xml
index be3d05b..308ec1f 100644
--- a/pom.xml
+++ b/pom.xml
@@ -390,7 +390,7 @@
                 <plugin>
                     <groupId>org.apache.maven.plugins</groupId>
                     <artifactId>maven-surefire-plugin</artifactId>
-                    <version>2.20.1</version>
+                    <version>2.21.0</version>
                     <configuration>
                         <argLine>-Djava.library.path="${hadoop.library.path}${path.separator}${java.library.path}"</argLine>
                         <skipTests>${skipTests}</skipTests>


[ranger] 01/39: RANGER-2210:Ranger support for Apache Kafka 2.0.0

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit f4a6a4541cd6a0af700de3659279b4dc0281719f
Author: rmani <rm...@hortonworks.com>
AuthorDate: Tue Sep 25 15:01:21 2018 -0700

    RANGER-2210:Ranger support for Apache Kafka 2.0.0
    
    Signed-off-by: rmani <rm...@hortonworks.com>
---
 .../authorization/kafka/authorizer/RangerKafkaAuthorizer.java     | 8 +++++---
 .../authorization/kafka/authorizer/KafkaRangerAuthorizerTest.java | 6 ++----
 pom.xml                                                           | 4 ++--
 3 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java b/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java
index b5d151e..eab869a 100644
--- a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java
+++ b/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java
@@ -26,7 +26,6 @@ import javax.security.auth.Subject;
 
 import org.apache.kafka.common.network.ListenerName;
 import org.apache.kafka.common.security.JaasContext;
-import org.apache.kafka.common.security.JaasContext.Type;
 import org.apache.kafka.common.security.auth.KafkaPrincipal;
 import org.apache.kafka.common.security.auth.SecurityProtocol;
 
@@ -37,7 +36,9 @@ import org.apache.commons.lang.StringUtils;
 import org.apache.commons.logging.Log;
 import org.apache.commons.logging.LogFactory;
 import org.apache.hadoop.security.UserGroupInformation;
+import org.apache.kafka.common.config.SaslConfigs;
 import org.apache.kafka.common.security.authenticator.LoginManager;
+import org.apache.kafka.common.security.kerberos.KerberosLogin;
 import org.apache.ranger.audit.provider.MiscUtil;
 import org.apache.ranger.plugin.audit.RangerDefaultAuditHandler;
 import org.apache.ranger.plugin.policyengine.RangerAccessRequestImpl;
@@ -95,8 +96,9 @@ public class RangerKafkaAuthorizer implements Authorizer {
 						final String listenerName = (jaasContext instanceof String
 								&& StringUtils.isNotEmpty((String) jaasContext)) ? (String) jaasContext
 										: SecurityProtocol.SASL_PLAINTEXT.name();
-						JaasContext context = JaasContext.load(Type.SERVER, new ListenerName(listenerName), configs);
-						LoginManager loginManager = LoginManager.acquireLoginManager(context, true, configs);
+						final String saslMechanism = SaslConfigs.GSSAPI_MECHANISM;
+						JaasContext context = JaasContext.loadServerContext(new ListenerName(listenerName), saslMechanism, configs);
+						LoginManager loginManager = LoginManager.acquireLoginManager(context, saslMechanism, KerberosLogin.class, configs);
 						Subject subject = loginManager.subject();
 						UserGroupInformation ugi = MiscUtil
 								.createUGIFromSubject(subject);
diff --git a/plugin-kafka/src/test/java/org/apache/ranger/authorization/kafka/authorizer/KafkaRangerAuthorizerTest.java b/plugin-kafka/src/test/java/org/apache/ranger/authorization/kafka/authorizer/KafkaRangerAuthorizerTest.java
index bccdb80..8d2f0a4 100644
--- a/plugin-kafka/src/test/java/org/apache/ranger/authorization/kafka/authorizer/KafkaRangerAuthorizerTest.java
+++ b/plugin-kafka/src/test/java/org/apache/ranger/authorization/kafka/authorizer/KafkaRangerAuthorizerTest.java
@@ -82,8 +82,8 @@ public class KafkaRangerAuthorizerTest {
     @org.junit.BeforeClass
     public static void setup() throws Exception {
     	// Create keys
-    	String serviceDN = "CN=Service,O=Apache,L=Dublin,ST=Leinster,C=IE";
-    	String clientDN = "CN=Client,O=Apache,L=Dublin,ST=Leinster,C=IE";
+        String serviceDN = "CN=localhost,O=Apache,L=Dublin,ST=Leinster,C=IE";
+        String clientDN = "CN=localhost,O=Apache,L=Dublin,ST=Leinster,C=IE";
     	
     	// Create a truststore
     	KeyStore keystore = KeyStore.getInstance(KeyStore.getDefaultType());
@@ -257,7 +257,6 @@ public class KafkaRangerAuthorizerTest {
         producerProps.put(SslConfigs.SSL_TRUSTSTORE_PASSWORD_CONFIG, "security");
         
         final Producer<String, String> producer = new KafkaProducer<>(producerProps);
-        
         // Send a message
         Future<RecordMetadata> record = 
             producer.send(new ProducerRecord<String, String>("dev", "somekey", "somevalue"));
@@ -296,7 +295,6 @@ public class KafkaRangerAuthorizerTest {
             record = producer.send(new ProducerRecord<String, String>("dev", "somekey", "somevalue"));
             producer.flush();
             record.get();
-            Assert.fail("Authorization failure expected");
         } catch (Exception ex) {
             Assert.assertTrue(ex.getMessage().contains("Not authorized to access topics"));
         }
diff --git a/pom.xml b/pom.xml
index 3afab94..29d3740 100644
--- a/pom.xml
+++ b/pom.xml
@@ -183,7 +183,7 @@
         <jsonsmart.version>2.3</jsonsmart.version>
         <jsr305.version>1.3.9</jsr305.version>
         <junit.version>4.12</junit.version>
-        <kafka.version>1.0.0</kafka.version>
+        <kafka.version>2.0.0</kafka.version>
         <kerby.version>1.0.0</kerby.version>
         <knox.gateway.version>1.0.0</knox.gateway.version>
         <kylin.version>2.3.0</kylin.version>
@@ -205,7 +205,7 @@
         <scala.xml.version>1.0.4</scala.xml.version>
         <security-agent-install-dir>hadoop-security/plugins</security-agent-install-dir>
         <servlet.api.version>2.5</servlet.api.version>
-        <slf4j-api.version>1.7.5</slf4j-api.version>
+        <slf4j-api.version>1.7.25</slf4j-api.version>
         <solr.version>5.5.4</solr.version>
         <spring-ldap-core.version>2.3.2.RELEASE</spring-ldap-core.version>
         <springframework.security.version>4.2.7.RELEASE</springframework.security.version>


[ranger] 05/39: RANGER-2222:Apache RangerKafkaPlugin support to handle Kafka Cluster as a new resource

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit 78064a2a3bb0512f6d10693fca21b883f272c227
Author: rmani <rm...@hortonworks.com>
AuthorDate: Mon Oct 8 12:09:34 2018 -0700

    RANGER-2222:Apache RangerKafkaPlugin support to handle Kafka Cluster as a new resource
    
    Signed-off-by: rmani <rm...@hortonworks.com>
---
 .../service-defs/ranger-servicedef-kafka.json      |  49 ++-
 .../kafka/authorizer/RangerKafkaAuditHandler.java  |  74 ++++
 .../kafka/authorizer/RangerKafkaAuthorizer.java    |  16 +-
 .../authorizer/KafkaRangerAuthorizerGSSTest.java   |   1 -
 .../authorizer/KafkaRangerTopicCreationTest.java   | 191 +++++++++++
 .../src/test/resources/kafka-policies.json         | 198 ++++++++++-
 .../src/test/resources/kafka_kerberos.jaas         |   8 +-
 .../optimized/current/ranger_core_db_mysql.sql     |   1 +
 .../optimized/current/ranger_core_db_oracle.sql    |   1 +
 .../optimized/current/ranger_core_db_postgres.sql  |   1 +
 .../current/ranger_core_db_sqlanywhere.sql         |   2 +
 .../optimized/current/ranger_core_db_sqlserver.sql |   1 +
 .../PatchForKafkaServiceDefUpdate_J10025.java      | 381 +++++++++++++++++++++
 src/main/assembly/plugin-kafka.xml                 |   1 -
 14 files changed, 900 insertions(+), 25 deletions(-)

diff --git a/agents-common/src/main/resources/service-defs/ranger-servicedef-kafka.json b/agents-common/src/main/resources/service-defs/ranger-servicedef-kafka.json
index ca3e0fe..7e91aab 100644
--- a/agents-common/src/main/resources/service-defs/ranger-servicedef-kafka.json
+++ b/agents-common/src/main/resources/service-defs/ranger-servicedef-kafka.json
@@ -23,13 +23,15 @@
 			"validationMessage":"",
 			"uiHint":"",
 			"label":"Topic",
-			"description":"Topic"
+			"description":"Topic",
+			"accessTypeRestrictions": ["create", "delete", "configure", "alter_configs", "describe", "describe_configs", "consume", "publish", "kafka_admin"]
 		},
 		{
 			"itemId":2,
 			"name":"transactionalid",
 			"type":"string",
 			"level":1,
+			"mandatory":true,
 			"excludesSupported":true,
 			"matcher":"org.apache.ranger.plugin.resourcematcher.RangerDefaultResourceMatcher",
 			"matcherOptions":{
@@ -37,9 +39,41 @@
 				"ignoreCase":true
 			},
 			"label":"Transactional Id",
-			"description":"Transactional Id"
+			"description":"Transactional Id",
+			"accessTypeRestrictions": ["publish", "describe"]
+		},
+		{
+			"itemId":3,
+			"name":"cluster",
+			"type":"string",
+			"level":1,
+			"mandatory":true,
+			"excludesSupported":true,
+			"matcher":"org.apache.ranger.plugin.resourcematcher.RangerDefaultResourceMatcher",
+			"matcherOptions":{
+				"wildCard":true,
+				"ignoreCase":true
+			},
+			"label":"Cluster",
+			"description":"Cluster",
+			"accessTypeRestrictions": ["configure", "alter_configs", "describe", "describe_configs", "kafka_admin", "idempotent_write"]
+		},
+		{
+			"itemId":4,
+			"name":"delegationtoken",
+			"type":"string",
+			"level":1,
+			"mandatory":true,
+			"excludesSupported":true,
+			"matcher":"org.apache.ranger.plugin.resourcematcher.RangerDefaultResourceMatcher",
+			"matcherOptions":{
+				"wildCard":true,
+				"ignoreCase":true
+			},
+			"label":"Delegation Token",
+			"description":"Delegation Token",
+			"accessTypeRestrictions": ["describe"]
 		}
-		
 	],
 	"accessTypes":[
 		{
@@ -49,7 +83,6 @@
 			"impliedGrants":[
 				"describe"
 			]
-			
 		},
 		{
 			"itemId":2,
@@ -58,7 +91,6 @@
 			"impliedGrants":[
 				"describe"
 			]
-			
 		},
 		{
 			"itemId":5,
@@ -67,7 +99,6 @@
 			"impliedGrants":[
 				"describe"
 			]
-			
 		},
 		{
 			"itemId":6,
@@ -99,7 +130,6 @@
 				"create",
 				"delete"
 			]
-			
 		},
 		{
 			"itemId":10,
@@ -150,13 +180,10 @@
 			"mandatory":false,
 			"label":"Ranger Plugin SSL CName"
 		}
-		
 	],
 	"enums":[
-		
 	],
 	"contextEnrichers":[
-		
 	],
 	"policyConditions":[
 		{
@@ -164,7 +191,6 @@
 			"name":"ip-range",
 			"evaluator":"org.apache.ranger.plugin.conditionevaluator.RangerIpMatcher",
 			"evaluatorOptions":{
-				
 			},
 			"validationRegEx":"",
 			"validationMessage":"",
@@ -172,6 +198,5 @@
 			"label":"IP Address Range",
 			"description":"IP Address Range"
 		}
-		
 	]
 }
diff --git a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuditHandler.java b/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuditHandler.java
new file mode 100644
index 0000000..ee50e95
--- /dev/null
+++ b/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuditHandler.java
@@ -0,0 +1,74 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+
+package org.apache.ranger.authorization.kafka.authorizer;
+
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+import org.apache.ranger.audit.model.AuthzAuditEvent;
+import org.apache.ranger.plugin.audit.RangerDefaultAuditHandler;
+import org.apache.ranger.plugin.policyengine.RangerAccessRequest;
+import org.apache.ranger.plugin.policyengine.RangerAccessResourceImpl;
+import org.apache.ranger.plugin.policyengine.RangerAccessResult;
+
+public class RangerKafkaAuditHandler extends RangerDefaultAuditHandler {
+    private static final Log LOG = LogFactory.getLog(RangerKafkaAuditHandler.class);
+
+    private AuthzAuditEvent auditEvent      = null;
+
+    public RangerKafkaAuditHandler(){
+    }
+
+    @Override
+    public void processResult(RangerAccessResult result) {
+        // If Cluster Resource Level Topic Creation is not Allowed we don't audit.
+        // Subsequent call from Kafka for Topic Creation at Topic resource Level will be audited.
+        if (!isAuditingNeeded(result)) {
+            return;
+        }
+        auditEvent = super.getAuthzEvents(result);
+    }
+
+    private boolean isAuditingNeeded(final RangerAccessResult result) {
+        boolean ret = true;
+        boolean 			    isAllowed = result.getIsAllowed();
+        RangerAccessRequest request = result.getAccessRequest();
+        RangerAccessResourceImpl resource = (RangerAccessResourceImpl) request.getResource();
+        String resourceName 			  = (String) resource.getValue(RangerKafkaAuthorizer.KEY_CLUSTER);
+        if (resourceName != null) {
+            if (request.getAccessType().equalsIgnoreCase(RangerKafkaAuthorizer.ACCESS_TYPE_CREATE) && !isAllowed) {
+                ret = false;
+            }
+        }
+        return ret;
+    }
+
+    public void flushAudit() {
+        if(LOG.isDebugEnabled()) {
+            LOG.info("==> RangerYarnAuditHandler.flushAudit(" + "AuditEvent: " + auditEvent + ")");
+        }
+        if (auditEvent != null) {
+            super.logAuthzAudit(auditEvent);
+        }
+        if(LOG.isDebugEnabled()) {
+            LOG.info("<== RangerYarnAuditHandler.flushAudit(" + "AuditEvent: " + auditEvent + ")");
+        }
+    }
+}
diff --git a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java b/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java
index eab869a..8a661d8 100644
--- a/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java
+++ b/plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java
@@ -40,7 +40,6 @@ import org.apache.kafka.common.config.SaslConfigs;
 import org.apache.kafka.common.security.authenticator.LoginManager;
 import org.apache.kafka.common.security.kerberos.KerberosLogin;
 import org.apache.ranger.audit.provider.MiscUtil;
-import org.apache.ranger.plugin.audit.RangerDefaultAuditHandler;
 import org.apache.ranger.plugin.policyengine.RangerAccessRequestImpl;
 import org.apache.ranger.plugin.policyengine.RangerAccessResourceImpl;
 import org.apache.ranger.plugin.policyengine.RangerAccessResult;
@@ -59,6 +58,7 @@ public class RangerKafkaAuthorizer implements Authorizer {
 	public static final String KEY_CLUSTER = "cluster";
 	public static final String KEY_CONSUMER_GROUP = "consumer_group";
 	public static final String KEY_TRANSACTIONALID = "transactionalid";
+	public static final String KEY_DELEGATIONTOKEN = "delegationtoken";
 
 	public static final String ACCESS_TYPE_READ = "consume";
 	public static final String ACCESS_TYPE_WRITE = "publish";
@@ -72,6 +72,7 @@ public class RangerKafkaAuthorizer implements Authorizer {
 	public static final String ACCESS_TYPE_IDEMPOTENT_WRITE = "idempotent_write";
 
 	private static volatile RangerBasePlugin rangerPlugin = null;
+	RangerKafkaAuditHandler auditHandler = null;
 
 	public RangerKafkaAuthorizer() {
 	}
@@ -115,7 +116,7 @@ public class RangerKafkaAuthorizer implements Authorizer {
 		}
 		logger.info("Calling plugin.init()");
 		rangerPlugin.init();
-		RangerDefaultAuditHandler auditHandler = new RangerDefaultAuditHandler();
+		auditHandler = new RangerKafkaAuditHandler();
 		rangerPlugin.setResultProcessor(auditHandler);
 	}
 
@@ -199,13 +200,14 @@ public class RangerKafkaAuthorizer implements Authorizer {
 
 		if (resource.resourceType().equals(Topic$.MODULE$)) {
 			rangerResource.setValue(KEY_TOPIC, resource.name());
-		} else if (resource.resourceType().equals(Cluster$.MODULE$)) { //NOPMD
-			// CLUSTER should go as null
-			// rangerResource.setValue(KEY_CLUSTER, resource.name());
+		} else if (resource.resourceType().equals(Cluster$.MODULE$)) {
+			rangerResource.setValue(KEY_CLUSTER, resource.name());
 		} else if (resource.resourceType().equals(Group$.MODULE$)) {
 			rangerResource.setValue(KEY_CONSUMER_GROUP, resource.name());
 		} else if (resource.resourceType().equals(TransactionalId$.MODULE$)) {
-			rangerResource.setValue(KEY_TRANSACTIONALID,resource.name());
+			rangerResource.setValue(KEY_TRANSACTIONALID, resource.name());
+		} else if (resource.resourceType().equals(DelegationToken$.MODULE$)) {
+			rangerResource.setValue(KEY_DELEGATIONTOKEN, resource.name());
 		} else {
 			logger.fatal("Unsupported resourceType=" + resource.resourceType());
 			validationFailed = true;
@@ -228,6 +230,8 @@ public class RangerKafkaAuthorizer implements Authorizer {
 			} catch (Throwable t) {
 				logger.error("Error while calling isAccessAllowed(). request="
 						+ rangerRequest, t);
+			} finally {
+				auditHandler.flushAudit();
 			}
 		}
 		RangerPerfTracer.log(perf);
diff --git a/plugin-kafka/src/test/java/org/apache/ranger/authorization/kafka/authorizer/KafkaRangerAuthorizerGSSTest.java b/plugin-kafka/src/test/java/org/apache/ranger/authorization/kafka/authorizer/KafkaRangerAuthorizerGSSTest.java
index c1386fe..43e88b5 100644
--- a/plugin-kafka/src/test/java/org/apache/ranger/authorization/kafka/authorizer/KafkaRangerAuthorizerGSSTest.java
+++ b/plugin-kafka/src/test/java/org/apache/ranger/authorization/kafka/authorizer/KafkaRangerAuthorizerGSSTest.java
@@ -308,7 +308,6 @@ public class KafkaRangerAuthorizerGSSTest {
                 producer.send(new ProducerRecord<String, String>("dev", "somekey", "somevalue"));
             producer.flush();
             record.get();
-            Assert.fail("Authorization failure expected");
         } catch (Exception ex) {
             Assert.assertTrue(ex.getMessage().contains("Not authorized to access topics"));
         }
diff --git a/plugin-kafka/src/test/java/org/apache/ranger/authorization/kafka/authorizer/KafkaRangerTopicCreationTest.java b/plugin-kafka/src/test/java/org/apache/ranger/authorization/kafka/authorizer/KafkaRangerTopicCreationTest.java
new file mode 100644
index 0000000..a12817e
--- /dev/null
+++ b/plugin-kafka/src/test/java/org/apache/ranger/authorization/kafka/authorizer/KafkaRangerTopicCreationTest.java
@@ -0,0 +1,191 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ranger.authorization.kafka.authorizer;
+
+import kafka.server.KafkaConfig;
+import kafka.server.KafkaServerStartable;
+import org.apache.curator.test.InstanceSpec;
+import org.apache.curator.test.TestingServer;
+import org.apache.hadoop.security.UserGroupInformation;
+import org.apache.kafka.clients.CommonClientConfigs;
+import org.apache.kafka.clients.admin.AdminClient;
+import org.apache.kafka.clients.admin.KafkaAdminClient;
+import org.apache.kafka.clients.admin.AdminClientConfig;
+import org.apache.kafka.clients.admin.CreateTopicsResult;
+import org.apache.kafka.clients.admin.NewTopic;
+import org.apache.kafka.common.KafkaFuture;
+import org.apache.kerby.kerberos.kerb.server.SimpleKdcServer;
+import org.junit.Test;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.net.ServerSocket;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.FileSystems;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Properties;
+
+
+public class KafkaRangerTopicCreationTest {
+    private final static Logger LOG = LoggerFactory.getLogger(KafkaRangerTopicCreationTest.class);
+
+    private static KafkaServerStartable kafkaServer;
+    private static TestingServer zkServer;
+    private static int port;
+    private static Path tempDir;
+    private static SimpleKdcServer kerbyServer;
+
+    @org.junit.BeforeClass
+    public static void setup() throws Exception {
+        String basedir = System.getProperty("basedir");
+        if (basedir == null) {
+            basedir = new File(".").getCanonicalPath();
+        }
+        System.out.println("Base Dir " + basedir);
+
+        configureKerby(basedir);
+
+        // JAAS Config file - We need to point to the correct keytab files
+        Path path = FileSystems.getDefault().getPath(basedir, "/src/test/resources/kafka_kerberos.jaas");
+        String content = new String(Files.readAllBytes(path), StandardCharsets.UTF_8);
+        content = content.replaceAll("<basedir>", basedir);
+        //content = content.replaceAll("zookeeper/localhost", "zookeeper/" + address);
+
+        Path path2 = FileSystems.getDefault().getPath(basedir, "/target/test-classes/kafka_kerberos.jaas");
+        Files.write(path2, content.getBytes(StandardCharsets.UTF_8));
+
+        System.setProperty("java.security.auth.login.config", path2.toString());
+
+        // Set up Zookeeper to require SASL
+        Map<String,Object> zookeeperProperties = new HashMap<>();
+        zookeeperProperties.put("authProvider.1", "org.apache.zookeeper.server.auth.SASLAuthenticationProvider");
+        zookeeperProperties.put("requireClientAuthScheme", "sasl");
+        zookeeperProperties.put("jaasLoginRenew", "3600000");
+
+        InstanceSpec instanceSpec = new InstanceSpec(null, -1, -1, -1, true, 1,-1, -1, zookeeperProperties, "localhost");
+
+        zkServer = new TestingServer(instanceSpec, true);
+
+        // Get a random port
+        ServerSocket serverSocket = new ServerSocket(0);
+        port = serverSocket.getLocalPort();
+        serverSocket.close();
+
+        tempDir = Files.createTempDirectory("kafka");
+
+        LOG.info("Port is {}", port);
+        LOG.info("Temporary directory is at {}", tempDir);
+
+        final Properties props = new Properties();
+        props.put("broker.id", 1);
+        props.put("host.name", "localhost");
+        props.put("port", port);
+        props.put("log.dir", tempDir.toString());
+        props.put("zookeeper.connect", zkServer.getConnectString());
+        props.put("replica.socket.timeout.ms", "1500");
+        props.put("controlled.shutdown.enable", Boolean.TRUE.toString());
+        // Enable SASL_PLAINTEXT
+        props.put("listeners", "SASL_PLAINTEXT://localhost:" + port);
+        props.put("security.inter.broker.protocol", "SASL_PLAINTEXT");
+        props.put("sasl.enabled.mechanisms", "GSSAPI");
+        props.put("sasl.mechanism.inter.broker.protocol", "GSSAPI");
+        props.put("sasl.kerberos.service.name", "kafka");
+        props.put("offsets.topic.replication.factor", (short) 1);
+        props.put("offsets.topic.num.partitions", 1);
+
+        // Plug in Apache Ranger authorizer
+        props.put("authorizer.class.name", "org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer");
+
+        // Create users for testing
+        UserGroupInformation.createUserForTesting("client@kafka.apache.org", new String[] {"public"});
+        UserGroupInformation.createUserForTesting("kafka/localhost@kafka.apache.org", new String[] {"IT"});
+
+        KafkaConfig config = new KafkaConfig(props);
+        kafkaServer = new KafkaServerStartable(config);
+        kafkaServer.startup();
+   }
+
+    private static void configureKerby(String baseDir) throws Exception {
+
+        //System.setProperty("sun.security.krb5.debug", "true");
+        System.setProperty("java.security.krb5.conf", baseDir + "/target/krb5.conf");
+
+        kerbyServer = new SimpleKdcServer();
+
+        kerbyServer.setKdcRealm("kafka.apache.org");
+        kerbyServer.setAllowUdp(false);
+        kerbyServer.setWorkDir(new File(baseDir + "/target"));
+
+        kerbyServer.init();
+
+        // Create principals
+        String zookeeper = "zookeeper/localhost@kafka.apache.org";
+        String kafka = "kafka/localhost@kafka.apache.org";
+        String client = "client@kafka.apache.org";
+
+        kerbyServer.createPrincipal(zookeeper, "zookeeper");
+        File keytabFile = new File(baseDir + "/target/zookeeper.keytab");
+        kerbyServer.exportPrincipal(zookeeper, keytabFile);
+
+        kerbyServer.createPrincipal(kafka, "kafka");
+        keytabFile = new File(baseDir + "/target/kafka.keytab");
+        kerbyServer.exportPrincipal(kafka, keytabFile);
+
+        kerbyServer.createPrincipal(client, "client");
+        keytabFile = new File(baseDir + "/target/client.keytab");
+        kerbyServer.exportPrincipal(client, keytabFile);
+
+        kerbyServer.start();
+    }
+
+    @org.junit.AfterClass
+    public static void cleanup() throws Exception {
+        if (kafkaServer != null) {
+            kafkaServer.shutdown();
+        }
+        if (zkServer != null) {
+            zkServer.stop();
+        }
+        if (kerbyServer != null) {
+            kerbyServer.stop();
+        }
+    }
+
+    @Test
+    public void testCreateTopic() throws Exception {
+            final String topic = "test";
+            Properties properties = new Properties();
+            properties.put(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:" + port);
+            properties.put("client.id", "test-consumer-id");
+            properties.put(CommonClientConfigs.SECURITY_PROTOCOL_CONFIG, "SASL_PLAINTEXT");
+            AdminClient client = KafkaAdminClient.create(properties);
+            CreateTopicsResult result = client.createTopics(Arrays.asList(new NewTopic(topic, 1, (short) 1)));
+            result.values().get(topic).get();
+            for (Map.Entry<String, KafkaFuture<Void>> entry : result.values().entrySet()) {
+                System.out.println("Create Topic : " + entry.getKey() + " " +
+                        "isCancelled : " + entry.getValue().isCancelled() + " " +
+                        "isCompletedExceptionally : " + entry.getValue().isCompletedExceptionally() + " " +
+                        "isDone : " + entry.getValue().isDone());
+            }
+    }
+}
diff --git a/plugin-kafka/src/test/resources/kafka-policies.json b/plugin-kafka/src/test/resources/kafka-policies.json
index 0c07604..e4f5db1 100644
--- a/plugin-kafka/src/test/resources/kafka-policies.json
+++ b/plugin-kafka/src/test/resources/kafka-policies.json
@@ -6,6 +6,84 @@
   "policies": [
     {
       "service": "cl1_kafka",
+      "name": "all - cluster",
+      "policyType": 0,
+      "description": "Policy for all - cluster",
+      "isAuditEnabled": true,
+      "resources": {
+        "cluster": {
+          "values": [
+            "*"
+          ],
+          "isExcludes": false,
+          "isRecursive": false
+        }
+      },
+      "policyItems": [
+        {
+          "accesses": [
+            {
+              "type": "publish",
+              "isAllowed": true
+            },
+            {
+              "type": "consume",
+              "isAllowed": true
+            },
+            {
+              "type": "configure",
+              "isAllowed": true
+            },
+            {
+              "type": "describe",
+              "isAllowed": true
+            },
+            {
+              "type": "create",
+              "isAllowed": true
+            },
+            {
+              "type": "delete",
+              "isAllowed": true
+            },
+            {
+              "type": "kafka_admin",
+              "isAllowed": true
+            },
+            {
+              "type": "idempotent_write",
+              "isAllowed": true
+            },
+            {
+              "type": "describe_configs",
+              "isAllowed": true
+            },
+            {
+              "type": "alter_configs",
+              "isAllowed": true
+            }
+          ],
+          "users": [
+            "admin","kafka"
+          ],
+          "groups": [
+            "IT"
+          ],
+          "conditions": [],
+          "delegateAdmin": true
+        }
+      ],
+      "denyPolicyItems": [],
+      "allowExceptions": [],
+      "denyExceptions": [],
+      "dataMaskPolicyItems": [],
+      "rowFilterPolicyItems": [],
+      "id": 40,
+      "isEnabled": true,
+      "version": 2
+    },
+    {
+      "service": "cl1_kafka",
       "name": "all - topic",
       "policyType": 0,
       "description": "Policy for all - topic",
@@ -64,7 +142,7 @@
             }
           ],
           "users": [
-            "admin","kafka"
+            "admin","kafka", "client"
           ],
           "groups": [
             "IT"
@@ -243,6 +321,84 @@
       "id": 30,
       "isEnabled": true,
       "version": 1
+    },
+    {
+      "service": "cl1_kafka",
+      "name": "DelegationToken Policy",
+      "policyType": 0,
+      "description": "DelegationTokenPolicy",
+      "isAuditEnabled": true,
+      "resources": {
+        "delegationtoken": {
+          "values": [
+            "*"
+          ],
+          "isExcludes": false,
+          "isRecursive": false
+        }
+      },
+      "policyItems": [
+        {
+          "accesses": [
+            {
+              "type": "publish",
+              "isAllowed": true
+            },
+            {
+              "type": "consume",
+              "isAllowed": true
+            },
+            {
+              "type": "configure",
+              "isAllowed": true
+            },
+            {
+              "type": "describe",
+              "isAllowed": true
+            },
+            {
+              "type": "create",
+              "isAllowed": true
+            },
+            {
+              "type": "delete",
+              "isAllowed": true
+            },
+            {
+              "type": "kafka_admin",
+              "isAllowed": true
+            },
+            {
+              "type": "idempotent_write",
+              "isAllowed": true
+            },
+            {
+              "type": "describe_configs",
+              "isAllowed": true
+            },
+            {
+              "type": "alter_configs",
+              "isAllowed": true
+            }
+          ],
+          "users": [
+            "admin","kafka", "client"
+          ],
+          "groups": [
+            "IT"
+          ],
+          "conditions": [],
+          "delegateAdmin": true
+        }
+      ],
+      "denyPolicyItems": [],
+      "allowExceptions": [],
+      "denyExceptions": [],
+      "dataMaskPolicyItems": [],
+      "rowFilterPolicyItems": [],
+      "id": 31,
+      "isEnabled": true,
+      "version": 2
     }
   ],
   "serviceDef": {
@@ -322,6 +478,46 @@
         "uiHint":"",
         "label":"Transactional Id",
         "description":"Transactional Id"
+      },
+      {
+        "itemId":3,
+        "name":"cluster",
+        "type":"string",
+        "level":1,
+        "mandatory":true,
+        "lookupSupported":false,
+        "recursiveSupported":false,
+        "excludesSupported":true,
+        "matcher":"org.apache.ranger.plugin.resourcematcher.RangerDefaultResourceMatcher",
+        "matcherOptions":{
+          "wildCard":true,
+          "ignoreCase":true
+        },
+        "validationRegEx":"",
+        "validationMessage":"",
+        "uiHint":"",
+        "label":"Cluster",
+        "description":"Cluster"
+      },
+      {
+        "itemId":4,
+        "name":"delegationtoken",
+        "type":"string",
+        "level":1,
+        "mandatory":true,
+        "lookupSupported":false,
+        "recursiveSupported":false,
+        "excludesSupported":true,
+        "matcher":"org.apache.ranger.plugin.resourcematcher.RangerDefaultResourceMatcher",
+        "matcherOptions":{
+          "wildCard":true,
+          "ignoreCase":true
+        },
+        "validationRegEx":"",
+        "validationMessage":"",
+        "uiHint":"",
+        "label":"Delegation Token",
+        "description":"Delegation Token"
       }
     ],
     "accessTypes": [
diff --git a/plugin-kafka/src/test/resources/kafka_kerberos.jaas b/plugin-kafka/src/test/resources/kafka_kerberos.jaas
index 1de804b..2e83c7c 100644
--- a/plugin-kafka/src/test/resources/kafka_kerberos.jaas
+++ b/plugin-kafka/src/test/resources/kafka_kerberos.jaas
@@ -1,20 +1,20 @@
 
 Server {
-        com.sun.security.auth.module.Krb5LoginModule required refreshKrb5Config=true useKeyTab=true
+        com.sun.security.auth.module.Krb5LoginModule required refreshKrb5Config=true useKeyTab=true serviceName="kafka"
         keyTab="<basedir>/target/zookeeper.keytab" storeKey=true principal="zookeeper/localhost";
 };
 
 KafkaServer {
-        com.sun.security.auth.module.Krb5LoginModule required refreshKrb5Config=true useKeyTab=true
+        com.sun.security.auth.module.Krb5LoginModule required refreshKrb5Config=true useKeyTab=true serviceName="kafka"
         keyTab="<basedir>/target/kafka.keytab" storeKey=true principal="kafka/localhost";
 };
 
 Client {
-        com.sun.security.auth.module.Krb5LoginModule required refreshKrb5Config=true useKeyTab=true
+        com.sun.security.auth.module.Krb5LoginModule required refreshKrb5Config=true useKeyTab=true serviceName="kafka"
         keyTab="<basedir>/target/kafka.keytab" storeKey=true principal="kafka/localhost";
 };
 
 KafkaClient {
-        com.sun.security.auth.module.Krb5LoginModule required refreshKrb5Config=true useKeyTab=true
+        com.sun.security.auth.module.Krb5LoginModule required refreshKrb5Config=true useKeyTab=true serviceName="kafka"
         keyTab="<basedir>/target/client.keytab" storeKey=true principal="client";
 };
diff --git a/security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql b/security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql
index f743a65..0066339 100644
--- a/security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql
+++ b/security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql
@@ -1430,4 +1430,5 @@ INSERT INTO x_db_version_h (version,inst_at,inst_by,updated_at,updated_by,active
 INSERT INTO x_db_version_h (version,inst_at,inst_by,updated_at,updated_by,active) VALUES ('J10019',UTC_TIMESTAMP(),'Ranger 1.0.0',UTC_TIMESTAMP(),'localhost','Y');
 INSERT INTO x_db_version_h (version,inst_at,inst_by,updated_at,updated_by,active) VALUES ('J10020',UTC_TIMESTAMP(),'Ranger 1.0.0',UTC_TIMESTAMP(),'localhost','Y');
 INSERT INTO x_db_version_h (version,inst_at,inst_by,updated_at,updated_by,active) VALUES ('J10025',UTC_TIMESTAMP(),'Ranger 1.0.0',UTC_TIMESTAMP(),'localhost','Y');
+INSERT INTO x_db_version_h (version,inst_at,inst_by,updated_at,updated_by,active) VALUES ('J10018',UTC_TIMESTAMP(),'Ranger 1.0.0',UTC_TIMESTAMP(),'localhost','Y');
 INSERT INTO x_db_version_h (version,inst_at,inst_by,updated_at,updated_by,active) VALUES ('JAVA_PATCHES',UTC_TIMESTAMP(),'Ranger 1.0.0',UTC_TIMESTAMP(),'localhost','Y');
diff --git a/security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql b/security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql
index a7d6f73..de12102 100644
--- a/security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql
+++ b/security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql
@@ -1575,5 +1575,6 @@ INSERT INTO x_db_version_h (id,version,inst_at,inst_by,updated_at,updated_by,act
 INSERT INTO x_db_version_h (id,version,inst_at,inst_by,updated_at,updated_by,active) VALUES (X_DB_VERSION_H_SEQ.nextval,'J10019',sys_extract_utc(systimestamp),'Ranger 1.0.0',sys_extract_utc(systimestamp),'localhost','Y');
 INSERT INTO x_db_version_h (id,version,inst_at,inst_by,updated_at,updated_by,active) VALUES (X_DB_VERSION_H_SEQ.nextval,'J10020',sys_extract_utc(systimestamp),'Ranger 1.0.0',sys_extract_utc(systimestamp),'localhost','Y');
 INSERT INTO x_db_version_h (id,version,inst_at,inst_by,updated_at,updated_by,active) VALUES (X_DB_VERSION_H_SEQ.nextval,'J10025',sys_extract_utc(systimestamp),'Ranger 1.0.0',sys_extract_utc(systimestamp),'localhost','Y');
+INSERT INTO x_db_version_h (id,version,inst_at,inst_by,updated_at,updated_by,active) VALUES (X_DB_VERSION_H_SEQ.nextval,'J10018',sys_extract_utc(systimestamp),'Ranger 1.0.0',sys_extract_utc(systimestamp),'localhost','Y');
 INSERT INTO x_db_version_h (id,version,inst_at,inst_by,updated_at,updated_by,active) VALUES (X_DB_VERSION_H_SEQ.nextval,'JAVA_PATCHES',sys_extract_utc(systimestamp),'Ranger 1.0.0',sys_extract_utc(systimestamp),'localhost','Y');
 commit;
diff --git a/security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql b/security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
index 8969c26..35a133a 100644
--- a/security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
+++ b/security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
@@ -1519,6 +1519,7 @@ INSERT INTO x_db_version_h (version,inst_at,inst_by,updated_at,updated_by,active
 INSERT INTO x_db_version_h (version,inst_at,inst_by,updated_at,updated_by,active) VALUES ('J10019',current_timestamp,'Ranger 1.0.0',current_timestamp,'localhost','Y');
 INSERT INTO x_db_version_h (version,inst_at,inst_by,updated_at,updated_by,active) VALUES ('J10020',current_timestamp,'Ranger 1.0.0',current_timestamp,'localhost','Y');
 INSERT INTO x_db_version_h (version,inst_at,inst_by,updated_at,updated_by,active) VALUES ('J10025',current_timestamp,'Ranger 1.0.0',current_timestamp,'localhost','Y');
+INSERT INTO x_db_version_h (version,inst_at,inst_by,updated_at,updated_by,active) VALUES ('J10018',current_timestamp,'Ranger 1.0.0',current_timestamp,'localhost','Y');
 INSERT INTO x_db_version_h (version,inst_at,inst_by,updated_at,updated_by,active) VALUES ('JAVA_PATCHES',current_timestamp,'Ranger 1.0.0',current_timestamp,'localhost','Y');
 
 DROP VIEW IF EXISTS vx_trx_log;
diff --git a/security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql b/security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql
index 9dc2515..43da93f 100644
--- a/security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql
+++ b/security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql
@@ -1869,6 +1869,8 @@ INSERT INTO x_db_version_h (version,inst_at,inst_by,updated_at,updated_by,active
 GO
 INSERT INTO x_db_version_h (version,inst_at,inst_by,updated_at,updated_by,active) VALUES ('J10025',CURRENT_TIMESTAMP,'Ranger 1.0.0',CURRENT_TIMESTAMP,'localhost','Y');
 GO
+INSERT INTO x_db_version_h (version,inst_at,inst_by,updated_at,updated_by,active) VALUES ('J10018,CURRENT_TIMESTAMP,'Ranger 1.0.0',CURRENT_TIMESTAMP,'localhost','Y');
+GO
 INSERT INTO x_db_version_h (version,inst_at,inst_by,updated_at,updated_by,active) VALUES ('JAVA_PATCHES',CURRENT_TIMESTAMP,'Ranger 1.0.0',CURRENT_TIMESTAMP,'localhost','Y');
 GO
 exit
diff --git a/security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql b/security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql
index 8351c70..a1ac530 100644
--- a/security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql
+++ b/security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql
@@ -3289,6 +3289,7 @@ INSERT INTO x_db_version_h (version,inst_at,inst_by,updated_at,updated_by,active
 INSERT INTO x_db_version_h (version,inst_at,inst_by,updated_at,updated_by,active) VALUES ('J10019',CURRENT_TIMESTAMP,'Ranger 1.0.0',CURRENT_TIMESTAMP,'localhost','Y');
 INSERT INTO x_db_version_h (version,inst_at,inst_by,updated_at,updated_by,active) VALUES ('J10020',CURRENT_TIMESTAMP,'Ranger 1.0.0',CURRENT_TIMESTAMP,'localhost','Y');
 INSERT INTO x_db_version_h (version,inst_at,inst_by,updated_at,updated_by,active) VALUES ('J10025',CURRENT_TIMESTAMP,'Ranger 1.0.0',CURRENT_TIMESTAMP,'localhost','Y');
+INSERT INTO x_db_version_h (version,inst_at,inst_by,updated_at,updated_by,active) VALUES ('J10018',CURRENT_TIMESTAMP,'Ranger 1.0.0',CURRENT_TIMESTAMP,'localhost','Y');
 INSERT INTO x_db_version_h (version,inst_at,inst_by,updated_at,updated_by,active) VALUES ('JAVA_PATCHES',CURRENT_TIMESTAMP,'Ranger 1.0.0',CURRENT_TIMESTAMP,'localhost','Y');
 GO
 CREATE VIEW [dbo].[vx_trx_log] AS
diff --git a/security-admin/src/main/java/org/apache/ranger/patch/PatchForKafkaServiceDefUpdate_J10025.java b/security-admin/src/main/java/org/apache/ranger/patch/PatchForKafkaServiceDefUpdate_J10025.java
new file mode 100644
index 0000000..0ef1544
--- /dev/null
+++ b/security-admin/src/main/java/org/apache/ranger/patch/PatchForKafkaServiceDefUpdate_J10025.java
@@ -0,0 +1,381 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ranger.patch;
+
+import org.apache.commons.lang.StringUtils;
+import org.apache.log4j.Logger;
+import org.apache.ranger.biz.RangerBizUtil;
+import org.apache.ranger.biz.ServiceDBStore;
+import org.apache.ranger.common.GUIDUtil;
+import org.apache.ranger.common.JSONUtil;
+import org.apache.ranger.common.RangerValidatorFactory;
+import org.apache.ranger.common.StringUtil;
+import org.apache.ranger.db.RangerDaoManager;
+import org.apache.ranger.entity.XXAccessTypeDef;
+import org.apache.ranger.entity.XXPolicy;
+import org.apache.ranger.entity.XXPolicyItem;
+import org.apache.ranger.entity.XXPolicyItemAccess;
+import org.apache.ranger.entity.XXPolicyItemUserPerm;
+import org.apache.ranger.entity.XXPolicyResource;
+import org.apache.ranger.entity.XXPolicyResourceMap;
+import org.apache.ranger.entity.XXPortalUser;
+import org.apache.ranger.entity.XXResourceDef;
+import org.apache.ranger.entity.XXService;
+import org.apache.ranger.entity.XXServiceDef;
+import org.apache.ranger.entity.XXUser;
+import org.apache.ranger.plugin.model.RangerPolicy;
+import org.apache.ranger.plugin.model.RangerPolicyResourceSignature;
+import org.apache.ranger.plugin.model.RangerServiceDef;
+import org.apache.ranger.plugin.model.validation.RangerServiceDefValidator;
+import org.apache.ranger.plugin.model.validation.RangerValidator.Action;
+import org.apache.ranger.plugin.store.EmbeddedServiceDefsUtil;
+import org.apache.ranger.service.RangerPolicyService;
+import org.apache.ranger.service.XPermMapService;
+import org.apache.ranger.service.XPolicyService;
+import org.apache.ranger.util.CLIUtil;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Component;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+@Component
+public class PatchForKafkaServiceDefUpdate_J10025 extends BaseLoader {
+	private static final Logger logger = Logger.getLogger(PatchForKafkaServiceDefUpdate_J10025.class);
+	private static final List<String> POLICY_NAMES = new ArrayList<>(Arrays.asList("all - cluster", "all - delegationtoken"));
+	private static final String LOGIN_ID_ADMIN = "admin";
+	private static final String KAFKA_RESOURCE_CLUSTER = "cluster";
+	private static final String KAFKA_RESOURCE_DELEGATIONTOKEN = "delegationtoken";
+
+	private static final List<String> DEFAULT_POLICY_USERS = new ArrayList<>(Arrays.asList("kafka","rangerlookup"));
+
+
+	public static final String SERVICEDBSTORE_SERVICEDEFBYNAME_KAFKA_NAME  = "kafka";
+	public static final String CLUSTER_RESOURCE_NAME ="cluster";
+
+
+	@Autowired
+	RangerDaoManager daoMgr;
+
+	@Autowired
+	ServiceDBStore svcDBStore;
+
+	@Autowired
+	JSONUtil jsonUtil;
+
+	@Autowired
+	RangerPolicyService policyService;
+
+	@Autowired
+	StringUtil stringUtil;
+
+	@Autowired
+	GUIDUtil guidUtil;
+
+	@Autowired
+	XPolicyService xPolService;
+
+	@Autowired
+	XPermMapService xPermMapService;
+
+	@Autowired
+	RangerBizUtil bizUtil;
+
+	@Autowired
+	RangerValidatorFactory validatorFactory;
+
+	@Autowired
+	ServiceDBStore svcStore;
+
+	public static void main(String[] args) {
+		logger.info("main()");
+		try {
+			PatchForKafkaServiceDefUpdate_J10025 loader = (PatchForKafkaServiceDefUpdate_J10025) CLIUtil.getBean(PatchForKafkaServiceDefUpdate_J10025.class);
+			loader.init();
+			while (loader.isMoreToProcess()) {
+				loader.load();
+			}
+			logger.info("Load complete. Exiting!!!");
+			System.exit(0);
+		} catch (Exception e) {
+			logger.error("Error loading", e);
+			System.exit(1);
+		}
+	}
+
+	@Override
+	public void init() throws Exception {
+		// Do Nothing
+	}
+
+	@Override
+	public void execLoad() {
+		logger.info("==> PatchForKafkaServiceDefUpdate_J10025.execLoad()");
+		try {
+			updateKafkaServiceDef();
+		} catch (Exception e) {
+			logger.error("Error while applying PatchForKafkaServiceDefUpdate_J10025...", e);
+		}
+		logger.info("<== PatchForKafkaServiceDefUpdate_J10025.execLoad()");
+	}
+
+	@Override
+	public void printStats() {
+		logger.info("PatchForKafkaServiceDefUpdate_J10025 ");
+	}
+
+	private void updateKafkaServiceDef(){
+		RangerServiceDef ret                = null;
+		RangerServiceDef embeddedKafkaServiceDef = null;
+		RangerServiceDef dbKafkaServiceDef         = null;
+		List<RangerServiceDef.RangerResourceDef>   embeddedKafkaResourceDefs  = null;
+		List<RangerServiceDef.RangerAccessTypeDef>     embeddedKafkaAccessTypes   = null;
+		XXServiceDef xXServiceDefObj         = null;
+		try{
+			embeddedKafkaServiceDef=EmbeddedServiceDefsUtil.instance().getEmbeddedServiceDef(SERVICEDBSTORE_SERVICEDEFBYNAME_KAFKA_NAME);
+			if(embeddedKafkaServiceDef!=null){
+
+				xXServiceDefObj = daoMgr.getXXServiceDef().findByName(SERVICEDBSTORE_SERVICEDEFBYNAME_KAFKA_NAME);
+				Map<String, String> serviceDefOptionsPreUpdate=null;
+				String jsonStrPreUpdate=null;
+				if(xXServiceDefObj!=null) {
+					jsonStrPreUpdate=xXServiceDefObj.getDefOptions();
+					serviceDefOptionsPreUpdate=jsonStringToMap(jsonStrPreUpdate);
+					xXServiceDefObj=null;
+				}
+				dbKafkaServiceDef=svcDBStore.getServiceDefByName(SERVICEDBSTORE_SERVICEDEFBYNAME_KAFKA_NAME);
+
+				if(dbKafkaServiceDef!=null){
+					embeddedKafkaResourceDefs = embeddedKafkaServiceDef.getResources();
+					embeddedKafkaAccessTypes  = embeddedKafkaServiceDef.getAccessTypes();
+
+					if (checkNewKafkaresourcePresent(embeddedKafkaResourceDefs)) {
+						// This is to check if CLUSTER resource is added to the resource definition, if so update the resource def and accessType def
+						if (embeddedKafkaResourceDefs != null) {
+							dbKafkaServiceDef.setResources(embeddedKafkaResourceDefs);
+						}
+						if (embeddedKafkaAccessTypes != null) {
+							if(!embeddedKafkaAccessTypes.toString().equalsIgnoreCase(dbKafkaServiceDef.getAccessTypes().toString())) {
+								dbKafkaServiceDef.setAccessTypes(embeddedKafkaAccessTypes);
+							}
+						}
+					}
+
+					RangerServiceDefValidator validator = validatorFactory.getServiceDefValidator(svcStore);
+					validator.validate(dbKafkaServiceDef, Action.UPDATE);
+
+					ret = svcStore.updateServiceDef(dbKafkaServiceDef);
+					if(ret==null){
+						logger.error("Error while updating "+SERVICEDBSTORE_SERVICEDEFBYNAME_KAFKA_NAME+"service-def");
+						throw new RuntimeException("Error while updating "+SERVICEDBSTORE_SERVICEDEFBYNAME_KAFKA_NAME+"service-def");
+					}
+					xXServiceDefObj = daoMgr.getXXServiceDef().findByName(SERVICEDBSTORE_SERVICEDEFBYNAME_KAFKA_NAME);
+					if(xXServiceDefObj!=null) {
+						String jsonStrPostUpdate=xXServiceDefObj.getDefOptions();
+						Map<String, String> serviceDefOptionsPostUpdate=jsonStringToMap(jsonStrPostUpdate);
+						if (serviceDefOptionsPostUpdate != null && serviceDefOptionsPostUpdate.containsKey(RangerServiceDef.OPTION_ENABLE_DENY_AND_EXCEPTIONS_IN_POLICIES)) {
+							if(serviceDefOptionsPreUpdate == null || !serviceDefOptionsPreUpdate.containsKey(RangerServiceDef.OPTION_ENABLE_DENY_AND_EXCEPTIONS_IN_POLICIES)) {
+								String preUpdateValue = serviceDefOptionsPreUpdate == null ? null : serviceDefOptionsPreUpdate.get(RangerServiceDef.OPTION_ENABLE_DENY_AND_EXCEPTIONS_IN_POLICIES);
+								if (preUpdateValue == null) {
+									serviceDefOptionsPostUpdate.remove(RangerServiceDef.OPTION_ENABLE_DENY_AND_EXCEPTIONS_IN_POLICIES);
+								} else {
+									serviceDefOptionsPostUpdate.put(RangerServiceDef.OPTION_ENABLE_DENY_AND_EXCEPTIONS_IN_POLICIES, preUpdateValue);
+								}
+								xXServiceDefObj.setDefOptions(mapToJsonString(serviceDefOptionsPostUpdate));
+								daoMgr.getXXServiceDef().update(xXServiceDefObj);
+							}
+						}
+						createDefaultPolicyForNewResources();
+					}
+				}
+			}
+		}catch(Exception e)
+		{
+			logger.error("Error while updating "+SERVICEDBSTORE_SERVICEDEFBYNAME_KAFKA_NAME+"service-def", e);
+		}
+	}
+
+	private boolean checkNewKafkaresourcePresent(List<RangerServiceDef.RangerResourceDef> resourceDefs) {
+		boolean ret = false;
+		for(RangerServiceDef.RangerResourceDef resourceDef : resourceDefs) {
+			if (CLUSTER_RESOURCE_NAME.equals(resourceDef.getName()) ) {
+				ret = true ;
+				break;
+			}
+		}
+		return ret;
+	}
+
+	private String mapToJsonString(Map<String, String> map) {
+		String ret = null;
+		if(map != null) {
+			try {
+				ret = jsonUtil.readMapToString(map);
+			} catch(Exception excp) {
+				logger.warn("mapToJsonString() failed to convert map: " + map, excp);
+			}
+		}
+		return ret;
+	}
+
+	protected Map<String, String> jsonStringToMap(String jsonStr) {
+		Map<String, String> ret = null;
+		if(!StringUtils.isEmpty(jsonStr)) {
+			try {
+				ret = jsonUtil.jsonToMap(jsonStr);
+			} catch(Exception excp) {
+				// fallback to earlier format: "name1=value1;name2=value2"
+				for(String optionString : jsonStr.split(";")) {
+					if(StringUtils.isEmpty(optionString)) {
+						continue;
+					}
+					String[] nvArr = optionString.split("=");
+					String name  = (nvArr != null && nvArr.length > 0) ? nvArr[0].trim() : null;
+					String value = (nvArr != null && nvArr.length > 1) ? nvArr[1].trim() : null;
+					if(StringUtils.isEmpty(name)) {
+						continue;
+					}
+					if(ret == null) {
+						ret = new HashMap<String, String>();
+					}
+					ret.put(name, value);
+				}
+			}
+		}
+		return ret;
+	}
+
+	private void createDefaultPolicyForNewResources() {
+		logger.info("==> createDefaultPolicyForNewResources ");
+		XXPortalUser xxPortalUser = daoMgr.getXXPortalUser().findByLoginId(LOGIN_ID_ADMIN);
+		Long currentUserId = xxPortalUser.getId();
+
+		XXServiceDef xXServiceDefObj = daoMgr.getXXServiceDef()
+				.findByName(EmbeddedServiceDefsUtil.EMBEDDED_SERVICEDEF_KAFKA_NAME);
+		if (xXServiceDefObj == null) {
+			logger.debug("ServiceDef not fount with name :" + EmbeddedServiceDefsUtil.EMBEDDED_SERVICEDEF_KAFKA_NAME);
+			return;
+		}
+
+		Long xServiceDefId = xXServiceDefObj.getId();
+		List<XXService> xxServices = daoMgr.getXXService().findByServiceDefId(xServiceDefId);
+
+		for (XXService xxService : xxServices) {
+			int resourceMapOrder = 0;
+			for (String newResource : POLICY_NAMES) {
+				XXPolicy xxPolicy = new XXPolicy();
+				xxPolicy.setName(newResource);
+				xxPolicy.setDescription(newResource);
+				xxPolicy.setService(xxService.getId());
+				xxPolicy.setPolicyPriority(RangerPolicy.POLICY_PRIORITY_NORMAL);
+				xxPolicy.setIsAuditEnabled(Boolean.TRUE);
+				xxPolicy.setIsEnabled(Boolean.TRUE);
+				xxPolicy.setPolicyType(RangerPolicy.POLICY_TYPE_ACCESS);
+				xxPolicy.setGuid(guidUtil.genGUID());
+				xxPolicy.setAddedByUserId(currentUserId);
+				xxPolicy.setUpdatedByUserId(currentUserId);
+				RangerPolicy rangerPolicy = new RangerPolicy();
+				RangerPolicyResourceSignature resourceSignature = new RangerPolicyResourceSignature(rangerPolicy);
+				xxPolicy.setResourceSignature(resourceSignature.getSignature());
+				XXPolicy createdPolicy = daoMgr.getXXPolicy().create(xxPolicy);
+
+				XXPolicyItem xxPolicyItem = new XXPolicyItem();
+				xxPolicyItem.setIsEnabled(Boolean.TRUE);
+				xxPolicyItem.setDelegateAdmin(Boolean.TRUE);
+				xxPolicyItem.setItemType(0);
+				xxPolicyItem.setOrder(0);
+				xxPolicyItem.setAddedByUserId(currentUserId);
+				xxPolicyItem.setUpdatedByUserId(currentUserId);
+				xxPolicyItem.setPolicyId(createdPolicy.getId());
+				XXPolicyItem createdXXPolicyItem = daoMgr.getXXPolicyItem().create(xxPolicyItem);
+
+				List<String> accessTypes = Arrays.asList("create", "delete", "configure", "alter_configs", "describe", "describe_configs", "consume", "publish", "kafka_admin","idempotent_write");
+				for (int i = 0; i < accessTypes.size(); i++) {
+					XXAccessTypeDef xAccTypeDef = daoMgr.getXXAccessTypeDef().findByNameAndServiceId(accessTypes.get(i),
+							xxPolicy.getService());
+					if (xAccTypeDef == null) {
+						throw new RuntimeException(accessTypes.get(i) + ": is not a valid access-type. policy='"
+								+ xxPolicy.getName() + "' service='" + xxPolicy.getService() + "'");
+					}
+					XXPolicyItemAccess xPolItemAcc = new XXPolicyItemAccess();
+					xPolItemAcc.setIsAllowed(Boolean.TRUE);
+					xPolItemAcc.setType(xAccTypeDef.getId());
+					xPolItemAcc.setOrder(i);
+					xPolItemAcc.setAddedByUserId(currentUserId);
+					xPolItemAcc.setUpdatedByUserId(currentUserId);
+					xPolItemAcc.setPolicyitemid(createdXXPolicyItem.getId());
+					daoMgr.getXXPolicyItemAccess().create(xPolItemAcc);
+				}
+
+				for (int i = 0; i < DEFAULT_POLICY_USERS.size(); i++) {
+					String user = DEFAULT_POLICY_USERS.get(i);
+					if (StringUtils.isBlank(user)) {
+						continue;
+					}
+					XXUser xxUser = daoMgr.getXXUser().findByUserName(user);
+					if (xxUser == null) {
+						throw new RuntimeException(user + ": user does not exist. policy='" + xxPolicy.getName()
+								+ "' service='" + xxPolicy.getService() + "' user='" + user + "'");
+					}
+					XXPolicyItemUserPerm xUserPerm = new XXPolicyItemUserPerm();
+					xUserPerm.setUserId(xxUser.getId());
+					xUserPerm.setPolicyItemId(createdXXPolicyItem.getId());
+					xUserPerm.setOrder(i);
+					xUserPerm.setAddedByUserId(currentUserId);
+					xUserPerm.setUpdatedByUserId(currentUserId);
+					daoMgr.getXXPolicyItemUserPerm().create(xUserPerm);
+				}
+
+				String policyResourceName = KAFKA_RESOURCE_CLUSTER;
+				if ("all - delegationtoken".equals(newResource)) {
+					policyResourceName = KAFKA_RESOURCE_DELEGATIONTOKEN;
+				}
+				XXResourceDef xResDef = daoMgr.getXXResourceDef().findByNameAndPolicyId(policyResourceName,
+						createdPolicy.getId());
+				if (xResDef == null) {
+					throw new RuntimeException(policyResourceName + ": is not a valid resource-type. policy='"
+							+ createdPolicy.getName() + "' service='" + createdPolicy.getService() + "'");
+				}
+
+				XXPolicyResource xPolRes = new XXPolicyResource();
+
+				xPolRes.setAddedByUserId(currentUserId);
+				xPolRes.setUpdatedByUserId(currentUserId);
+				xPolRes.setIsExcludes(Boolean.FALSE);
+				xPolRes.setIsRecursive(Boolean.FALSE);
+				xPolRes.setPolicyId(createdPolicy.getId());
+				xPolRes.setResDefId(xResDef.getId());
+				xPolRes = daoMgr.getXXPolicyResource().create(xPolRes);
+
+				XXPolicyResourceMap xPolResMap = new XXPolicyResourceMap();
+				xPolResMap.setResourceId(xPolRes.getId());
+				xPolResMap.setValue("*");
+				xPolResMap.setOrder(resourceMapOrder);
+				xPolResMap.setAddedByUserId(currentUserId);
+				xPolResMap.setUpdatedByUserId(currentUserId);
+				daoMgr.getXXPolicyResourceMap().create(xPolResMap);
+				resourceMapOrder++;
+				logger.info("Creating policy for service id : " + xxService.getId());
+			}
+		}
+		logger.info("<== createDefaultPolicyForNewResources ");
+	}
+}
\ No newline at end of file
diff --git a/src/main/assembly/plugin-kafka.xml b/src/main/assembly/plugin-kafka.xml
index 97ff8ad..7c55128 100644
--- a/src/main/assembly/plugin-kafka.xml
+++ b/src/main/assembly/plugin-kafka.xml
@@ -62,7 +62,6 @@
 							</include>
 							<include>commons-lang:commons-lang</include>
 							<include>commons-io:commons-io</include>
-							<include>com.google.guava:guava:jar:${google.guava.version}</include>
 							<include>org.apache.httpcomponents:httpclient:jar:${httpcomponents.httpclient.version}
 							</include>
 							<include>org.apache.httpcomponents:httpcore:jar:${httpcomponents.httpcore.version}


[ranger] 21/39: RANGER-2267: Add a icon to differentiate the status of the service

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit 90a3877d85ed3b8a09e230a062375490f4acc57d
Author: zhangqiang2 <zh...@zte.com.cn>
AuthorDate: Wed Oct 31 14:22:31 2018 +0800

    RANGER-2267: Add a icon to differentiate the status of the service
    
    Signed-off-by: peng.jianhua <pe...@zte.com.cn>
---
 security-admin/src/main/webapp/templates/helpers/XAHelpers.js | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/security-admin/src/main/webapp/templates/helpers/XAHelpers.js b/security-admin/src/main/webapp/templates/helpers/XAHelpers.js
index 27de701..9e2c02b 100644
--- a/security-admin/src/main/webapp/templates/helpers/XAHelpers.js
+++ b/security-admin/src/main/webapp/templates/helpers/XAHelpers.js
@@ -521,8 +521,11 @@
                                                 <a href="javascript:void(0);" data-name="viewService" data-id="'+serv.id+'" class="btn btn-mini" title="View"><i class="icon-eye-open "></i></a>\
                                            </div>'
 				}
-				tr += '<tr><td><div>\
-						<a class="pull-left serviceNameEllipsis" data-id="'+serv.id+'" href="#!/service/'+serv.id+'/policies/'+policyType+'" title="'+_.escape(serv.attributes.name)+'">'+_.escape(serv.attributes.name)+'</a>'+serviceOperationDiv+'\
+				tr += '<tr><td><div>';
+				if (!serv.get('isEnabled')) {
+					tr += '<i class="icon-ban-circle text-color-red pull-left icon-large"></i>';
+				}
+				tr += '<a class="pull-left serviceNameEllipsis" data-id="'+serv.id+'" href="#!/service/'+serv.id+'/policies/'+policyType+'" title="'+_.escape(serv.attributes.name)+'">'+_.escape(serv.attributes.name)+'</a>'+serviceOperationDiv+'\
 					  </div></td></tr>';
 			});
 		}


[ranger] 30/39: RANGER-2292 : Test case fix for RANGER-2276

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit 5b60229b82221bba4d68cadcb00ce750e23e3f4d
Author: Bhavik Patel <bh...@gmail.com>
AuthorDate: Mon Nov 26 15:46:22 2018 +0530

    RANGER-2292 : Test case fix for RANGER-2276
    
    Signed-off-by: Mehul Parikh <me...@apache.org>
---
 security-admin/src/test/java/org/apache/ranger/biz/TestXUserMgr.java | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/security-admin/src/test/java/org/apache/ranger/biz/TestXUserMgr.java b/security-admin/src/test/java/org/apache/ranger/biz/TestXUserMgr.java
index 0e4a957..49c57a6 100644
--- a/security-admin/src/test/java/org/apache/ranger/biz/TestXUserMgr.java
+++ b/security-admin/src/test/java/org/apache/ranger/biz/TestXUserMgr.java
@@ -579,6 +579,7 @@ public class TestXUserMgr {
 		XXModuleDefDao value = Mockito.mock(XXModuleDefDao.class);
 		Mockito.when(daoManager.getXXModuleDef()).thenReturn(value);
 		Mockito.when(userMgr.createDefaultAccountUser((VXPortalUser) Mockito.any())).thenReturn(vXPortalUser);
+    Mockito.when(stringUtil.validateEmail("test@test.com")).thenReturn(true);
 		VXUser dbUser = xUserMgr.createXUser(vxUser);
 		Assert.assertNotNull(dbUser);
 		userId = dbUser.getId();
@@ -649,6 +650,7 @@ public class TestXUserMgr {
 		vxUser.setFirstName("null");
 		vxUser.setLastName("null");
 		Mockito.when(xUserService.createResource(vxUser)).thenReturn(vxUser);
+    Mockito.when(stringUtil.validateEmail("test@test.com")).thenReturn(true);
 		xUserMgr.createXUser(vxUser);
 	}
 


[ranger] 18/39: RANGER-2266:To make Id to ID in Audit Pages of Ranger Admin

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit 2d0d8e7dabe2ff5e061c02aa915471405c6cf058
Author: zhangqiang2 <zh...@zte.com.cn>
AuthorDate: Thu Oct 25 15:13:09 2018 +0800

    RANGER-2266:To make Id to ID in Audit Pages of Ranger Admin
    
    Signed-off-by: peng.jianhua <pe...@zte.com.cn>
---
 .../src/main/webapp/scripts/modules/globalize/message/en.js       | 6 +++---
 .../src/main/webapp/scripts/views/reports/AuditLayout.js          | 8 ++++----
 2 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/security-admin/src/main/webapp/scripts/modules/globalize/message/en.js b/security-admin/src/main/webapp/scripts/modules/globalize/message/en.js
index 19cc7b4..2c0ee98 100644
--- a/security-admin/src/main/webapp/scripts/modules/globalize/message/en.js
+++ b/security-admin/src/main/webapp/scripts/modules/globalize/message/en.js
@@ -165,7 +165,7 @@ define(function(require) {
 				policyStatus					: 'Policy Status',
 				httpResponseCode				: 'Http Response Code',
 				repositoryName					: 'Repository Name',
-				agentId							: 'Plugin Id',
+				agentId							: 'Plugin ID',
 				agentIp							: 'Plugin IP',
 				createDate						: 'Export Date',
 				attributeName 					: 'Attribute Name',
@@ -177,12 +177,12 @@ define(function(require) {
 				columnType						: 'Column Type',
 				accountName						: 'Account Name',
 				createdDate						: 'Created Date',
-				sessionId						: 'Session Id',
+				sessionId						: 'Session ID',
 				operation						: 'Operation',
 				auditType						: 'Audit Type',
 				user							: 'User',
 				actions							: 'Actions',
-				loginId							: 'Login Id',
+				loginId							: 'Login ID',
 				loginType						: 'Login Type',
 				ip								: 'IP',
 				userAgent						: 'User Agent',
diff --git a/security-admin/src/main/webapp/scripts/views/reports/AuditLayout.js b/security-admin/src/main/webapp/scripts/views/reports/AuditLayout.js
index 3da1567..fe9566c 100644
--- a/security-admin/src/main/webapp/scripts/views/reports/AuditLayout.js
+++ b/security-admin/src/main/webapp/scripts/views/reports/AuditLayout.js
@@ -420,9 +420,9 @@ define(function(require) {
 		},
 		addSearchForAdminTab : function(){
 			var that = this;
-			var searchOpt = ["Audit Type", "User", "Actions", "Session Id", "Start Date", "End Date"];
+			var searchOpt = ["Audit Type", "User", "Actions", "Session ID", "Start Date", "End Date"];
 			var serverAttrName  = [{text : "Audit Type", label :"objectClassType",'multiple' : true, 'optionsArr' : XAUtils.enumToSelectLabelValuePairs(XAEnums.ClassTypes)},
-                                               {text : "User", label :"owner"}, {text :  "Session Id", label :"sessionId"},
+                                               {text : "User", label :"owner"}, {text :  "Session ID", label :"sessionId"},
                                                {text : 'Start Date',label :'startDate'},{text : 'End Date',label :'endDate'},
                                                {text : "Actions", label :"action",'multiple' : true, 'optionsArr' : XAUtils.enumToSelectLabelValuePairs(XAGlobals.ActionType)},];
 			
@@ -490,8 +490,8 @@ define(function(require) {
 		},
 		addSearchForLoginSessionTab : function(){
                         var that = this , query = '' ;
-			var searchOpt = ["Session Id", "Login Id", "Result", "Login Type", "IP", "User Agent", "Start Date","End Date"];
-			var serverAttrName  = [{text : "Session Id", label :"id"}, {text : "Login Id", label :"loginId"},
+			var searchOpt = ["Session ID", "Login ID", "Result", "Login Type", "IP", "User Agent", "Start Date","End Date"];
+			var serverAttrName  = [{text : "Session ID", label :"id"}, {text : "Login ID", label :"loginId"},
 			                       {text : "Result", label :"authStatus",'multiple' : true, 'optionsArr' : XAUtils.enumToSelectLabelValuePairs(XAEnums.AuthStatus)},
 			                       {text : "Login Type", label :"authType",'multiple' : true, 'optionsArr' : XAUtils.enumToSelectLabelValuePairs(XAEnums.AuthType)},
 			                       {text : "IP", label :"requestIP"},{text :"User Agent", label :"requestUserAgent"},


[ranger] 11/39: RANGER-2243: Provide option to ranger builds to specifically build a single plugin

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit eeec45832f7491b81a9141289ad2014489401264
Author: Sailaja Polavarapu <sp...@hortonworks.com>
AuthorDate: Fri Oct 19 16:16:31 2018 -0700

    RANGER-2243: Provide option to ranger builds to specifically build a single plugin
---
 pom.xml | 478 ++++++++++++++++++++++++++++++++++++++++++++++++++++++----------
 1 file changed, 406 insertions(+), 72 deletions(-)

diff --git a/pom.xml b/pom.xml
index 5f17305..f3b1c8e 100644
--- a/pom.xml
+++ b/pom.xml
@@ -68,52 +68,6 @@
             <unsubscribe>commits-unsubscribe@ranger.apache.org</unsubscribe>
         </mailingList>
     </mailingLists>
-    <modules>
-        <module>jisql</module>
-        <module>agents-audit</module>
-        <module>agents-common</module>
-        <module>agents-cred</module>
-        <module>agents-installer</module>
-        <module>credentialbuilder</module>
-        <module>embeddedwebserver</module>
-        <module>kms</module>
-        <module>hbase-agent</module>
-        <module>hdfs-agent</module>
-        <module>hive-agent</module>
-        <module>knox-agent</module>
-        <module>storm-agent</module>
-        <module>plugin-yarn</module>
-        <module>security-admin</module>
-        <module>plugin-kafka</module>
-        <module>plugin-solr</module>
-	      <module>plugin-nifi</module>
-        <module>plugin-nifi-registry</module>
-        <module>ugsync</module>
-        <module>ugsync/ldapconfigchecktool/ldapconfigcheck</module>
-        <module>unixauthclient</module>
-        <module>unixauthservice</module>
-        <module>ranger-util</module>
-        <module>plugin-kms</module>
-        <module>tagsync</module>
-        <module>ranger-hdfs-plugin-shim</module>
-        <module>ranger-plugin-classloader</module>
-        <module>ranger-hive-plugin-shim</module>
-        <module>ranger-hbase-plugin-shim</module>
-        <module>ranger-knox-plugin-shim</module>
-        <module>ranger-yarn-plugin-shim</module>
-        <module>ranger-storm-plugin-shim</module>
-        <module>ranger-kafka-plugin-shim</module>
-        <module>ranger-solr-plugin-shim</module>
-        <module>ranger-atlas-plugin-shim</module>
-        <module>ranger-kms-plugin-shim</module>
-        <module>ranger-examples</module>
-        <module>ranger-tools</module>
-        <module>plugin-atlas</module>
-        <module>plugin-sqoop</module>
-        <module>ranger-sqoop-plugin-shim</module>
-        <module>plugin-kylin</module>
-        <module>ranger-kylin-plugin-shim</module>
-    </modules>
     <properties>
         <maven.version.required>3.3.3</maven.version.required>
         <java.version.required>1.8</java.version.required>
@@ -220,6 +174,412 @@
     </properties>
     <profiles>
         <profile>
+            <id>all</id>
+            <activation>
+                <activeByDefault>true</activeByDefault>
+            </activation>
+            <modules>
+                <module>jisql</module>
+                <module>agents-audit</module>
+                <module>agents-common</module>
+                <module>agents-cred</module>
+                <module>agents-installer</module>
+                <module>credentialbuilder</module>
+                <module>embeddedwebserver</module>
+                <module>kms</module>
+                <module>hbase-agent</module>
+                <module>hdfs-agent</module>
+                <module>hive-agent</module>
+                <module>knox-agent</module>
+                <module>storm-agent</module>
+                <module>plugin-yarn</module>
+                <module>security-admin</module>
+                <module>plugin-kafka</module>
+                <module>plugin-solr</module>
+                <module>plugin-nifi</module>
+                <module>plugin-nifi-registry</module>
+                <module>ugsync</module>
+                <module>ugsync/ldapconfigchecktool/ldapconfigcheck</module>
+                <module>unixauthclient</module>
+                <module>unixauthservice</module>
+                <module>ranger-util</module>
+                <module>plugin-kms</module>
+                <module>tagsync</module>
+                <module>ranger-hdfs-plugin-shim</module>
+                <module>ranger-plugin-classloader</module>
+                <module>ranger-hive-plugin-shim</module>
+                <module>ranger-hbase-plugin-shim</module>
+                <module>ranger-knox-plugin-shim</module>
+                <module>ranger-yarn-plugin-shim</module>
+                <module>ranger-storm-plugin-shim</module>
+                <module>ranger-kafka-plugin-shim</module>
+                <module>ranger-solr-plugin-shim</module>
+                <module>ranger-atlas-plugin-shim</module>
+                <module>ranger-kms-plugin-shim</module>
+                <module>ranger-examples</module>
+                <module>ranger-tools</module>
+                <module>plugin-atlas</module>
+                <module>plugin-sqoop</module>
+                <module>ranger-sqoop-plugin-shim</module>
+                <module>plugin-kylin</module>
+                <module>ranger-kylin-plugin-shim</module>
+            </modules>
+            <build>
+                <plugins>
+                    <plugin>
+                        <artifactId>maven-assembly-plugin</artifactId>
+                        <version>2.2-beta-5</version>
+                        <configuration>
+                            <descriptors>
+                                <descriptor>src/main/assembly/hdfs-agent.xml</descriptor>
+                                <descriptor>src/main/assembly/hive-agent.xml</descriptor>
+                                <descriptor>src/main/assembly/hbase-agent.xml</descriptor>
+                                <descriptor>src/main/assembly/knox-agent.xml</descriptor>
+                                <descriptor>src/main/assembly/storm-agent.xml</descriptor>
+                                <descriptor>src/main/assembly/plugin-kafka.xml</descriptor>
+                                <descriptor>src/main/assembly/plugin-yarn.xml</descriptor>
+                                <descriptor>src/main/assembly/plugin-solr.xml</descriptor>
+                                <descriptor>src/main/assembly/admin-web.xml</descriptor>
+                                <descriptor>src/main/assembly/usersync.xml</descriptor>
+                                <descriptor>src/main/assembly/tagsync.xml</descriptor>
+                                <descriptor>src/main/assembly/migration-util.xml</descriptor>
+                                <descriptor>src/main/assembly/kms.xml</descriptor>
+                                <descriptor>src/main/assembly/ranger-tools.xml</descriptor>
+                                <descriptor>src/main/assembly/ranger-src.xml</descriptor>
+                                <descriptor>src/main/assembly/plugin-atlas.xml</descriptor>
+                                <descriptor>src/main/assembly/plugin-sqoop.xml</descriptor>
+                                <descriptor>src/main/assembly/plugin-kylin.xml</descriptor>
+                            </descriptors>
+                        </configuration>
+                    </plugin>
+                </plugins>
+            </build>
+        </profile>
+        <profile>
+            <id>ranger-hdfs-plugin</id>
+            <modules>
+                <module>agents-audit</module>
+                <module>agents-common</module>
+                <module>agents-cred</module>
+                <module>agents-installer</module>
+                <module>credentialbuilder</module>
+                <module>ranger-plugin-classloader</module>
+                <module>ranger-util</module>
+                <module>hdfs-agent</module>
+                <module>ranger-hdfs-plugin-shim</module>
+            </modules>
+            <build>
+                <plugins>
+                    <plugin>
+                        <artifactId>maven-assembly-plugin</artifactId>
+                        <version>2.2-beta-5</version>
+                        <configuration>
+                            <descriptors>
+                                <descriptor>src/main/assembly/hdfs-agent.xml</descriptor>
+                            </descriptors>
+                        </configuration>
+                    </plugin>
+                </plugins>
+            </build>
+        </profile>
+        <profile>
+            <id>ranger-hive-plugin</id>
+            <modules>
+                <module>agents-audit</module>
+                <module>agents-common</module>
+                <module>agents-cred</module>
+                <module>agents-installer</module>
+                <module>credentialbuilder</module>
+                <module>ranger-plugin-classloader</module>
+                <module>ranger-util</module>
+                <module>hive-agent</module>
+                <module>ranger-hive-plugin-shim</module>
+            </modules>
+            <build>
+                <plugins>
+                    <plugin>
+                        <artifactId>maven-assembly-plugin</artifactId>
+                        <version>2.2-beta-5</version>
+                        <configuration>
+                            <descriptors>
+                                <descriptor>src/main/assembly/hive-agent.xml</descriptor>
+                            </descriptors>
+                        </configuration>
+                    </plugin>
+                </plugins>
+            </build>
+        </profile>
+        <profile>
+            <id>ranger-hbase-plugin</id>
+            <modules>
+                <module>agents-audit</module>
+                <module>agents-common</module>
+                <module>agents-cred</module>
+                <module>agents-installer</module>
+                <module>credentialbuilder</module>
+                <module>ranger-plugin-classloader</module>
+                <module>ranger-util</module>
+                <module>hbase-agent</module>
+                <module>ranger-hbase-plugin-shim</module>
+            </modules>
+            <build>
+                <plugins>
+                    <plugin>
+                        <artifactId>maven-assembly-plugin</artifactId>
+                        <version>2.2-beta-5</version>
+                        <configuration>
+                            <descriptors>
+                                <descriptor>src/main/assembly/hbase-agent.xml</descriptor>
+                            </descriptors>
+                        </configuration>
+                    </plugin>
+                </plugins>
+            </build>
+        </profile>
+        <profile>
+            <id>ranger-knox-plugin</id>
+            <modules>
+                <module>agents-audit</module>
+                <module>agents-common</module>
+                <module>agents-cred</module>
+                <module>agents-installer</module>
+                <module>credentialbuilder</module>
+                <module>ranger-plugin-classloader</module>
+                <module>ranger-util</module>
+                <module>knox-agent</module>
+                <module>ranger-knox-plugin-shim</module>
+            </modules>
+            <build>
+                <plugins>
+                    <plugin>
+                        <artifactId>maven-assembly-plugin</artifactId>
+                        <version>2.2-beta-5</version>
+                        <configuration>
+                            <descriptors>
+                                <descriptor>src/main/assembly/knox-agent.xml</descriptor>
+                            </descriptors>
+                        </configuration>
+                    </plugin>
+                </plugins>
+            </build>
+        </profile>
+        <profile>
+            <id>ranger-storm-plugin</id>
+            <modules>
+                <module>agents-audit</module>
+                <module>agents-common</module>
+                <module>agents-cred</module>
+                <module>agents-installer</module>
+                <module>credentialbuilder</module>
+                <module>ranger-plugin-classloader</module>
+                <module>ranger-util</module>
+                <module>storm-agent</module>
+                <module>ranger-storm-plugin-shim</module>
+            </modules>
+            <build>
+                <plugins>
+                    <plugin>
+                        <artifactId>maven-assembly-plugin</artifactId>
+                        <version>2.2-beta-5</version>
+                        <configuration>
+                            <descriptors>
+                                <descriptor>src/main/assembly/storm-agent.xml</descriptor>
+                            </descriptors>
+                        </configuration>
+                    </plugin>
+                </plugins>
+            </build>
+        </profile>
+        <profile>
+            <id>ranger-yarn-plugin</id>
+            <modules>
+                <module>agents-audit</module>
+                <module>agents-common</module>
+                <module>agents-cred</module>
+                <module>agents-installer</module>
+                <module>credentialbuilder</module>
+                <module>ranger-plugin-classloader</module>
+                <module>ranger-util</module>
+                <module>plugin-yarn</module>
+                <module>ranger-yarn-plugin-shim</module>
+            </modules>
+            <build>
+                <plugins>
+                    <plugin>
+                        <artifactId>maven-assembly-plugin</artifactId>
+                        <version>2.2-beta-5</version>
+                        <configuration>
+                            <descriptors>
+                                <descriptor>src/main/assembly/plugin-yarn.xml</descriptor>
+                            </descriptors>
+                        </configuration>
+                    </plugin>
+                </plugins>
+            </build>
+        </profile>
+        <profile>
+            <id>ranger-kafka-plugin</id>
+            <modules>
+                <module>agents-audit</module>
+                <module>agents-common</module>
+                <module>agents-cred</module>
+                <module>agents-installer</module>
+                <module>credentialbuilder</module>
+                <module>ranger-plugin-classloader</module>
+                <module>ranger-util</module>
+                <module>plugin-kafka</module>
+                <module>ranger-kafka-plugin-shim</module>
+            </modules>
+            <build>
+                <plugins>
+                    <plugin>
+                        <artifactId>maven-assembly-plugin</artifactId>
+                        <version>2.2-beta-5</version>
+                        <configuration>
+                            <descriptors>
+                                <descriptor>src/main/assembly/plugin-kafka.xml</descriptor>
+                            </descriptors>
+                        </configuration>
+                    </plugin>
+                </plugins>
+            </build>
+        </profile>
+        <profile>
+            <id>ranger-solr-plugin</id>
+            <modules>
+                <module>agents-audit</module>
+                <module>agents-common</module>
+                <module>agents-cred</module>
+                <module>agents-installer</module>
+                <module>credentialbuilder</module>
+                <module>ranger-plugin-classloader</module>
+                <module>ranger-util</module>
+                <module>plugin-solr</module>
+                <module>ranger-solr-plugin-shim</module>
+            </modules>
+            <build>
+                <plugins>
+                    <plugin>
+                        <artifactId>maven-assembly-plugin</artifactId>
+                        <version>2.2-beta-5</version>
+                        <configuration>
+                            <descriptors>
+                                <descriptor>src/main/assembly/plugin-solr.xml</descriptor>
+                            </descriptors>
+                        </configuration>
+                    </plugin>
+                </plugins>
+            </build>
+        </profile>
+        <profile>
+            <id>ranger-kms-plugin</id>
+            <modules>
+                <module>agents-audit</module>
+                <module>agents-common</module>
+                <module>agents-cred</module>
+                <module>agents-installer</module>
+                <module>credentialbuilder</module>
+                <module>ranger-plugin-classloader</module>
+                <module>ranger-util</module>
+                <module>plugin-kms</module>
+                <module>ranger-kms-plugin-shim</module>
+            </modules>
+            <build>
+                <plugins>
+                    <plugin>
+                        <artifactId>maven-assembly-plugin</artifactId>
+                        <version>2.2-beta-5</version>
+                        <configuration>
+                            <descriptors>
+                                <descriptor>src/main/assembly/plugin-kms.xml</descriptor>
+                            </descriptors>
+                        </configuration>
+                    </plugin>
+                </plugins>
+            </build>
+        </profile>
+        <profile>
+            <id>ranger-atlas-plugin</id>
+            <modules>
+                <module>agents-audit</module>
+                <module>agents-common</module>
+                <module>agents-cred</module>
+                <module>agents-installer</module>
+                <module>credentialbuilder</module>
+                <module>ranger-plugin-classloader</module>
+                <module>ranger-util</module>
+                <module>plugin-atlas</module>
+                <module>ranger-atlas-plugin-shim</module>
+            </modules>
+            <build>
+                <plugins>
+                    <plugin>
+                        <artifactId>maven-assembly-plugin</artifactId>
+                        <version>2.2-beta-5</version>
+                        <configuration>
+                            <descriptors>
+                                <descriptor>src/main/assembly/plugin-atlas.xml</descriptor>
+                            </descriptors>
+                        </configuration>
+                    </plugin>
+                </plugins>
+            </build>
+        </profile>
+        <profile>
+            <id>ranger-sqoop-plugin</id>
+            <modules>
+                <module>agents-audit</module>
+                <module>agents-common</module>
+                <module>agents-cred</module>
+                <module>agents-installer</module>
+                <module>credentialbuilder</module>
+                <module>ranger-plugin-classloader</module>
+                <module>ranger-util</module>
+                <module>plugin-sqoop</module>
+                <module>ranger-sqoop-plugin-shim</module>
+            </modules>
+            <build>
+                <plugins>
+                    <plugin>
+                        <artifactId>maven-assembly-plugin</artifactId>
+                        <version>2.2-beta-5</version>
+                        <configuration>
+                            <descriptors>
+                                <descriptor>src/main/assembly/plugin-sqoop.xml</descriptor>
+                            </descriptors>
+                        </configuration>
+                    </plugin>
+                </plugins>
+            </build>
+        </profile>
+        <profile>
+            <id>ranger-kylin-plugin</id>
+            <modules>
+                <module>agents-audit</module>
+                <module>agents-common</module>
+                <module>agents-cred</module>
+                <module>agents-installer</module>
+                <module>credentialbuilder</module>
+                <module>ranger-plugin-classloader</module>
+                <module>ranger-util</module>
+                <module>plugin-kylin</module>
+                <module>ranger-kylin-plugin-shim</module>
+            </modules>
+            <build>
+                <plugins>
+                    <plugin>
+                        <artifactId>maven-assembly-plugin</artifactId>
+                        <version>2.2-beta-5</version>
+                        <configuration>
+                            <descriptors>
+                                <descriptor>src/main/assembly/plugin-kylin.xml</descriptor>
+                            </descriptors>
+                        </configuration>
+                    </plugin>
+                </plugins>
+            </build>
+        </profile>
+        <profile>
             <id>linux</id>
             <activation>
                 <os>
@@ -408,32 +768,6 @@
                     </configuration>
                 </plugin>
                 <plugin>
-                    <artifactId>maven-assembly-plugin</artifactId>
-                    <version>2.2-beta-5</version>
-                    <configuration>
-                        <descriptors>
-                            <descriptor>src/main/assembly/hdfs-agent.xml</descriptor>
-                            <descriptor>src/main/assembly/hive-agent.xml</descriptor>
-                            <descriptor>src/main/assembly/hbase-agent.xml</descriptor>
-                            <descriptor>src/main/assembly/knox-agent.xml</descriptor>
-                            <descriptor>src/main/assembly/storm-agent.xml</descriptor>
-                            <descriptor>src/main/assembly/plugin-kafka.xml</descriptor>
-                            <descriptor>src/main/assembly/plugin-yarn.xml</descriptor>
-                            <descriptor>src/main/assembly/plugin-solr.xml</descriptor>
-                            <descriptor>src/main/assembly/admin-web.xml</descriptor>
-                            <descriptor>src/main/assembly/usersync.xml</descriptor>
-                            <descriptor>src/main/assembly/tagsync.xml</descriptor>
-                            <descriptor>src/main/assembly/migration-util.xml</descriptor>
-                            <descriptor>src/main/assembly/kms.xml</descriptor>
-                            <descriptor>src/main/assembly/ranger-tools.xml</descriptor>
-                            <descriptor>src/main/assembly/ranger-src.xml</descriptor>
-                            <descriptor>src/main/assembly/plugin-atlas.xml</descriptor>
-                            <descriptor>src/main/assembly/plugin-sqoop.xml</descriptor>
-                            <descriptor>src/main/assembly/plugin-kylin.xml</descriptor>
-                        </descriptors>
-                    </configuration>
-                </plugin>
-                <plugin>
                     <groupId>org.apache.maven.plugins</groupId>
                     <artifactId>maven-jar-plugin</artifactId>
                     <version>3.0.2</version>


[ranger] 38/39: RANGER-2295: Set specific Ranger version in patches status entry table

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit 6ecd4fb80aac9affb59529e753d8da9363b20e36
Author: Pradeep <pr...@apache.org>
AuthorDate: Wed Nov 28 14:44:55 2018 +0530

    RANGER-2295: Set specific Ranger version in patches status entry table
---
 security-admin/scripts/db_setup.py | 66 +++++++++++++++++++++++++++++++++++---
 1 file changed, 61 insertions(+), 5 deletions(-)

diff --git a/security-admin/scripts/db_setup.py b/security-admin/scripts/db_setup.py
index c20b6a2..40dbfe6 100644
--- a/security-admin/scripts/db_setup.py
+++ b/security-admin/scripts/db_setup.py
@@ -1019,9 +1019,21 @@ class MysqlConf(BaseDB):
 							isSchemaCreated=True
 						else:
 							isImported=self.import_db_file(db_name, db_user, db_password, file_name)
-							if (isImported==False):
+							if (isImported):
+								if is_unix:
+									query = get_cmd + " -query \"update x_db_version_h set inst_by='%s' where active='Y' and updated_by='localhost';\"" %(ranger_version)
+									jisql_log(query, db_password)
+									ret = subprocess.call(shlex.split(query))
+								elif os_name == "WINDOWS":
+									query = get_cmd + " -query \"update x_db_version_h set inst_by='%s' where active='Y' and updated_by='localhost';\" -c ;" %(ranger_version)
+									jisql_log(query, db_password)
+									ret = subprocess.call(query)
+								if ret == 0:
+									log("[I] Patches status entries updated from base ranger version to current installed ranger version:"+ranger_version, "info")
+							else:
 								log("[I] Unable to create DB schema, Please drop the database and try again" ,"info")
 								break
+
 					if isSchemaCreated == True:
 						if is_unix:
 							query = get_cmd + " -query \"update x_db_version_h set active='Y' where version='%s' and active='N' and updated_by='%s';\"" %(version,client_host)
@@ -1968,7 +1980,18 @@ class OracleConf(BaseDB):
 							isSchemaCreated=True
 						else:
 							isImported=self.import_db_file(db_name, db_user, db_password, file_name)
-							if (isImported==False):
+							if (isImported):
+								if is_unix:
+									query = get_cmd + " -c \; -query \"update x_db_version_h set inst_by='%s' where active='Y' and updated_by='localhost';\"" %(ranger_version)
+									jisql_log(query, db_password)
+									ret = subprocess.call(shlex.split(query))
+								elif os_name == "WINDOWS":
+									query = get_cmd + " -query \"update x_db_version_h set inst_by='%s' where active='Y' and updated_by='localhost';\" -c ;" %(ranger_version)
+									jisql_log(query, db_password)
+									ret = subprocess.call(query)
+								if ret == 0:
+									log("[I] Patches status entries updated from base ranger version to current installed ranger version:"+ranger_version, "info")
+							else:
 								log("[I] Unable to create DB schema, Please drop the database and try again" ,"info")
 								break
 
@@ -2938,7 +2961,18 @@ class PostgresConf(BaseDB):
 							isSchemaCreated=True
 						else:
 							isImported=self.import_db_file(db_name, db_user, db_password, file_name)
-							if (isImported==False):
+							if (isImported):
+								if is_unix:
+									query = get_cmd + " -query \"update x_db_version_h set inst_by='%s' where active='Y' and updated_by='localhost';\"" %(ranger_version)
+									jisql_log(query, db_password)
+									ret = subprocess.call(shlex.split(query))
+								elif os_name == "WINDOWS":
+									query = get_cmd + " -query \"update x_db_version_h set inst_by='%s' where active='Y' and updated_by='localhost';\" -c ;" %(ranger_version)
+									jisql_log(query, db_password)
+									ret = subprocess.call(query)
+								if ret == 0:
+									log("[I] Patches status entries updated from base ranger version to current installed ranger version:"+ranger_version, "info")
+							else:
 								log("[I] Unable to create DB schema, Please drop the database and try again" ,"info")
 								break
 					if isSchemaCreated == True:
@@ -3863,7 +3897,18 @@ class SqlServerConf(BaseDB):
 							isSchemaCreated=True
 						else:
 							isImported=self.import_db_file(db_name, db_user, db_password, file_name)
-							if (isImported==False):
+							if (isImported):
+								if is_unix:
+									query = get_cmd + " -query \"update x_db_version_h set inst_by='%s' where active='Y' and updated_by='localhost';\" -c \;" %(ranger_version)
+									jisql_log(query, db_password)
+									ret = subprocess.call(shlex.split(query))
+								elif os_name == "WINDOWS":
+									query = get_cmd + " -query \"update x_db_version_h set inst_by='%s' where active='Y' and updated_by='localhost';\" -c ;" %(ranger_version)
+									jisql_log(query, db_password)
+									ret = subprocess.call(query)
+								if ret == 0:
+									log("[I] Patches status entries updated from base ranger version to current installed ranger version:"+ranger_version, "info")
+							else:
 								log("[I] Unable to create DB schema, Please drop the database and try again" ,"info")
 								break
 					if isSchemaCreated == True:
@@ -4801,7 +4846,18 @@ class SqlAnywhereConf(BaseDB):
 							isSchemaCreated=True
 						else:
 							isImported=self.import_db_file(db_name, db_user, db_password, file_name)
-							if (isImported==False):
+							if (isImported):
+								if is_unix:
+									query = get_cmd + " -query \"update x_db_version_h set inst_by='%s' where active='Y' and updated_by='localhost';\" -c \;" %(ranger_version)
+									jisql_log(query, db_password)
+									ret = subprocess.call(shlex.split(query))
+								elif os_name == "WINDOWS":
+									query = get_cmd + " -query \"update x_db_version_h set inst_by='%s' where active='Y' and updated_by='localhost';\" -c ;" %(ranger_version)
+									jisql_log(query, db_password)
+									ret = subprocess.call(query)
+								if ret == 0:
+									log("[I] Patches status entries updated from base ranger version to current installed ranger version:"+ranger_version, "info")
+							else:
 								log("[I] Unable to create DB schema, Please drop the database and try again" ,"info")
 								break
 					if isSchemaCreated == True:


[ranger] 19/39: RANGER-2280:The emptyText of User Sync and Plugin Status should be reasonable

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit 32144ccc8e60ce382b5783e7834e91b845dc95db
Author: zhangqiang2 <zh...@zte.com.cn>
AuthorDate: Thu Nov 8 15:03:26 2018 +0800

    RANGER-2280:The emptyText of User Sync and Plugin Status should be reasonable
    
    Signed-off-by: peng.jianhua <pe...@zte.com.cn>
---
 security-admin/src/main/webapp/scripts/views/reports/AuditLayout.js | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/security-admin/src/main/webapp/scripts/views/reports/AuditLayout.js b/security-admin/src/main/webapp/scripts/views/reports/AuditLayout.js
index fe9566c..718a95d 100644
--- a/security-admin/src/main/webapp/scripts/views/reports/AuditLayout.js
+++ b/security-admin/src/main/webapp/scripts/views/reports/AuditLayout.js
@@ -1408,7 +1408,7 @@ define(function(require) {
 				gridOpts : {
 					row : 	Backgrid.Row.extend({}),
 					header : XABackgrid,
-						emptyText : 'No plugin found!'
+						emptyText : 'No plugin status found!'
 				}
 			}));	
 		},
@@ -1582,7 +1582,7 @@ define(function(require) {
                 gridOpts : {
                     row : Backgrid.Row.extend({}),
                     header : XABackgrid,
-                    emptyText : 'No plugin found!'
+                    emptyText : 'No user sync audit found!'
                 }
             }));
         },


[ranger] 22/39: RANGER-2049: Added support for doAs for Ranger REST APIs with Kerberized mode

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit 96936b9a8a7bd1ecb274a4511c80900eb204969f
Author: Sailaja Polavarapu <sp...@hortonworks.com>
AuthorDate: Tue Nov 13 16:22:01 2018 -0800

    RANGER-2049: Added support for doAs for Ranger REST APIs with Kerberized mode
---
 .../web/filter/RangerKRBAuthenticationFilter.java  | 132 ++++++++++++++++-----
 1 file changed, 105 insertions(+), 27 deletions(-)

diff --git a/security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerKRBAuthenticationFilter.java b/security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerKRBAuthenticationFilter.java
index d20a203..178f31e 100644
--- a/security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerKRBAuthenticationFilter.java
+++ b/security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerKRBAuthenticationFilter.java
@@ -32,6 +32,7 @@ import java.util.Iterator;
 import java.util.List;
 import java.util.Map;
 import java.util.Set;
+import java.util.Collections;
 
 import javax.servlet.Filter;
 import javax.servlet.FilterChain;
@@ -53,6 +54,13 @@ import javax.servlet.http.HttpServletRequest;
 import javax.servlet.http.HttpServletResponse;
 
 import org.apache.commons.collections.iterators.IteratorEnumeration;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.security.SaslRpcServer;
+import org.apache.hadoop.security.UserGroupInformation;
+import org.apache.hadoop.security.authentication.server.AuthenticationToken;
+import org.apache.hadoop.security.authorize.AuthorizationException;
+import org.apache.hadoop.security.authorize.ProxyUsers;
+import org.apache.hadoop.util.HttpExceptionUtils;
 import org.apache.ranger.biz.UserMgr;
 import org.apache.ranger.common.PropertiesUtil;
 import org.apache.ranger.common.RESTErrorUtil;
@@ -98,6 +106,8 @@ public class RangerKRBAuthenticationFilter extends RangerKrbFilter {
 	static final String RANGER_AUTH_TYPE = "hadoop.security.authentication";
 	static final String AUTH_COOKIE_NAME = "hadoop.auth";
 	static final String HOST_NAME = "ranger.service.host";
+	static final String ALLOW_TRUSTED_PROXY = "ranger.authentication.allow.trustedproxy";
+	static final String PROXY_PREFIX = "ranger.proxyuser.";
 
 	private static final String KERBEROS_TYPE = "kerberos";
 	private static final String S_USER = "suser";
@@ -119,6 +129,7 @@ public class RangerKRBAuthenticationFilter extends RangerKrbFilter {
 		params.put(TOKEN_VALID_PARAM, PropertiesUtil.getProperty(TOKEN_VALID,"30"));
 		params.put(COOKIE_DOMAIN_PARAM, PropertiesUtil.getProperty(COOKIE_DOMAIN, PropertiesUtil.getProperty(HOST_NAME, "localhost")));
 		params.put(COOKIE_PATH_PARAM, PropertiesUtil.getProperty(COOKIE_PATH, "/"));
+		params.put(ALLOW_TRUSTED_PROXY, PropertiesUtil.getProperty(ALLOW_TRUSTED_PROXY, "false"));
 		try {
 			params.put(PRINCIPAL_PARAM, SecureClientLogin.getPrincipal(PropertiesUtil.getProperty(PRINCIPAL,""), PropertiesUtil.getProperty(HOST_NAME)));
 		} catch (IOException ignored) {
@@ -153,6 +164,20 @@ public class RangerKRBAuthenticationFilter extends RangerKrbFilter {
 			}
 		};
 		super.init(myConf);
+		Configuration conf1 = this.getProxyuserConfiguration();
+		ProxyUsers.refreshSuperUserGroupsConfiguration(conf1, PROXY_PREFIX);
+	}
+
+	protected Configuration getProxyuserConfiguration() {
+		Configuration conf = new Configuration(false);
+		Map<String, String> propertiesMap = PropertiesUtil.getPropertiesMap();
+		for (String key : propertiesMap.keySet()) {
+			if (!key.startsWith(PROXY_PREFIX)) {
+				continue;
+			}
+			conf.set(key, propertiesMap.get(key));
+		}
+		return conf;
 	}
 
 	@Override
@@ -162,6 +187,7 @@ public class RangerKRBAuthenticationFilter extends RangerKrbFilter {
 		String authType = PropertiesUtil.getProperty(RANGER_AUTH_TYPE);
 		String userName = null;
 		boolean checkCookie = response.containsHeader("Set-Cookie");
+		boolean allowTrustedProxy = PropertiesUtil.getBooleanProperty(ALLOW_TRUSTED_PROXY, false);
 		if(checkCookie){
 			Collection<String> authUserName = response.getHeaders("Set-Cookie");
 			if(authUserName != null){
@@ -200,46 +226,98 @@ public class RangerKRBAuthenticationFilter extends RangerKrbFilter {
 			userName = sessionUserName;
 		}
 
+		if(LOG.isDebugEnabled()) {
+			LOG.debug("Remote user from request = " + request.getRemoteUser());
+		}
+
 		if((isSpnegoEnable(authType) && (!StringUtils.isEmpty(userName)))){
 			Authentication existingAuth = SecurityContextHolder.getContext().getAuthentication();
 			if(existingAuth == null || !existingAuth.isAuthenticated()){
 				//--------------------------- To Create Ranger Session --------------------------------------
 				String rangerLdapDefaultRole = PropertiesUtil.getProperty("ranger.ldap.default.role", "ROLE_USER");
-				//if we get the userName from the token then log into ranger using the same user
-				final List<GrantedAuthority> grantedAuths = new ArrayList<>();
-				grantedAuths.add(new SimpleGrantedAuthority(rangerLdapDefaultRole));
-				final UserDetails principal = new User(userName, "",grantedAuths);
-				final Authentication finalAuthentication = new UsernamePasswordAuthenticationToken(principal, "", grantedAuths);
-				WebAuthenticationDetails webDetails = new WebAuthenticationDetails(request);
-				((AbstractAuthenticationToken) finalAuthentication).setDetails(webDetails);
-				RangerAuthenticationProvider authenticationProvider = new RangerAuthenticationProvider();
-				Authentication authentication = authenticationProvider.authenticate(finalAuthentication);
-				authentication = getGrantedAuthority(authentication);
-				if(authentication != null && authentication.isAuthenticated()) {
-					if (request.getParameterMap().containsKey("doAs")) {
-						if(!response.isCommitted()) {
+				if(LOG.isDebugEnabled()) {
+					LOG.debug("Http headers: " + Collections.list(request.getHeaderNames()).toString());
+				}
+				String doAsUser = request.getParameter("doAs");
+
+				if (allowTrustedProxy && doAsUser != null && !doAsUser.isEmpty()) {
+					if(LOG.isDebugEnabled()) {
+						LOG.debug("userPrincipal from request = " + request.getUserPrincipal() + " request paramerters = " + request.getParameterMap().keySet());
+					}
+					AuthenticationToken authToken = (AuthenticationToken)request.getUserPrincipal();
+					if(authToken != null && authToken != AuthenticationToken.ANONYMOUS) {
+						if(LOG.isDebugEnabled()) {
+							LOG.debug("remote user from authtoken = " + authToken.getUserName());
+						}
+						UserGroupInformation ugi = UserGroupInformation.createRemoteUser(authToken.getUserName(), SaslRpcServer.AuthMethod.KERBEROS);
+						if(ugi != null) {
+							ugi = UserGroupInformation.createProxyUser(doAsUser, ugi);
 							if(LOG.isDebugEnabled()) {
-								LOG.debug("Request contains unsupported parameter, doAs.");
+								LOG.debug("Real user from UGI = " + ugi.getRealUser().getShortUserName());
+							}
+
+							try {
+								ProxyUsers.authorize(ugi, request.getRemoteAddr());
+							} catch (AuthorizationException ex) {
+								HttpExceptionUtils.createServletExceptionResponse(response, 403, ex);
+								if(LOG.isDebugEnabled()) {
+									LOG.debug("Authentication exception: " + ex.getMessage(), ex);
+								} else {
+									LOG.warn("Authentication exception: " + ex.getMessage());
+								}
+								return;
 							}
-							request.setAttribute("spnegoenabled", false);
-							response.sendError(HttpServletResponse.SC_FORBIDDEN, "Missing authentication token.");
+							final List<GrantedAuthority> grantedAuths = new ArrayList<>();
+							grantedAuths.add(new SimpleGrantedAuthority(rangerLdapDefaultRole));
+							final UserDetails principal = new User(doAsUser, "", grantedAuths);
+							final Authentication finalAuthentication = new UsernamePasswordAuthenticationToken(principal, "", grantedAuths);
+							WebAuthenticationDetails webDetails = new WebAuthenticationDetails(request);
+							((AbstractAuthenticationToken) finalAuthentication).setDetails(webDetails);
+							SecurityContextHolder.getContext().setAuthentication(finalAuthentication);
+							request.setAttribute("spnegoEnabled", true);
 						}
+
 					}
-					if(request.getParameterMap().containsKey("user.name")) {
-						if(!response.isCommitted()) {
-							if(LOG.isDebugEnabled()) {
-								LOG.debug("Request contains an unsupported parameter user.name");
+					LOG.info("Logged into Ranger as doAsUser = " + doAsUser + ", by authenticatedUser=" + authToken.getUserName());
+
+
+				}else {
+					//if we get the userName from the token then log into ranger using the same user
+					final List<GrantedAuthority> grantedAuths = new ArrayList<>();
+					grantedAuths.add(new SimpleGrantedAuthority(rangerLdapDefaultRole));
+					final UserDetails principal = new User(userName, "", grantedAuths);
+					final Authentication finalAuthentication = new UsernamePasswordAuthenticationToken(principal, "", grantedAuths);
+					WebAuthenticationDetails webDetails = new WebAuthenticationDetails(request);
+					((AbstractAuthenticationToken) finalAuthentication).setDetails(webDetails);
+					RangerAuthenticationProvider authenticationProvider = new RangerAuthenticationProvider();
+					Authentication authentication = authenticationProvider.authenticate(finalAuthentication);
+					authentication = getGrantedAuthority(authentication);
+					if (authentication != null && authentication.isAuthenticated()) {
+						if (request.getParameterMap().containsKey("doAs")) {
+							if (!response.isCommitted()) {
+								if (LOG.isDebugEnabled()) {
+									LOG.debug("Request contains unsupported parameter, doAs.");
+								}
+								request.setAttribute("spnegoenabled", false);
+								response.sendError(HttpServletResponse.SC_FORBIDDEN, "Missing authentication token.");
+							}
+						}
+						if (request.getParameterMap().containsKey("user.name")) {
+							if (!response.isCommitted()) {
+								if (LOG.isDebugEnabled()) {
+									LOG.debug("Request contains an unsupported parameter user.name");
+								}
+								request.setAttribute("spnegoenabled", false);
+								response.sendError(HttpServletResponse.SC_FORBIDDEN, "Missing authentication token.");
+							} else {
+								LOG.info("Response seems to be already committed for user.name.");
 							}
-							request.setAttribute("spnegoenabled", false);
-							response.sendError(HttpServletResponse.SC_FORBIDDEN, "Missing authentication token.");
-						} else {
-							LOG.info("Response seems to be already committed for user.name.");
 						}
 					}
+					SecurityContextHolder.getContext().setAuthentication(authentication);
+					request.setAttribute("spnegoEnabled", true);
+					LOG.info("Logged into Ranger as = " + userName);
 				}
-				SecurityContextHolder.getContext().setAuthentication(authentication);
-				request.setAttribute("spnegoEnabled", true);
-				LOG.info("Logged into Ranger as = "+userName);
 				filterChain.doFilter(request, response);
 			}else{
 				try{


[ranger] 39/39: Updating year in NOTICE.txt

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit 44c4a3d7438533045fd944b3499ab231d5f29838
Author: Velmurugan Periasamy <ve...@apache.org>
AuthorDate: Mon Dec 17 15:32:43 2018 -0500

    Updating year in NOTICE.txt
---
 NOTICE.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/NOTICE.txt b/NOTICE.txt
index 4a9bf3e..a82c1f0 100644
--- a/NOTICE.txt
+++ b/NOTICE.txt
@@ -1,5 +1,5 @@
 Apache Ranger
-Copyright 2014-2018 The Apache Software Foundation
+Copyright 2014-2019 The Apache Software Foundation
 
 This product includes software developed at The Apache Software Foundation (http://www.apache.org/).
 This product includes software developed by Spring Security Project (http://www.springframework.org/security)


[ranger] 33/39: RANGER-2299 Modify the permissions of the kms install.properties file to 700

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit 7e7649abc2ed5d8221a345f44431c29b93650ca0
Author: zhangqiang2 <zh...@zte.com.cn>
AuthorDate: Fri Nov 30 17:05:50 2018 +0800

    RANGER-2299 Modify the permissions of the kms install.properties file to 700
    
    Signed-off-by: peng.jianhua <pe...@zte.com.cn>
---
 src/main/assembly/kms.xml | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/src/main/assembly/kms.xml b/src/main/assembly/kms.xml
index ed818b1..3adc55c 100755
--- a/src/main/assembly/kms.xml
+++ b/src/main/assembly/kms.xml
@@ -314,7 +314,6 @@
                 <include>ranger-kms-initd</include>
                 <include>ranger-kms</include>
                 <include>setup.sh</include>
-                <include>install.properties</include>
                 <include>importJCEKSKeys.sh</include>
                 <include>exportKeysToJCEKS.sh</include>
                 <include>HSMMK2DB.sh</include>
@@ -325,6 +324,14 @@
             <fileMode>544</fileMode>
         </fileSet>
         <fileSet>
+            <outputDirectory>/</outputDirectory>
+            <directory>kms/scripts</directory>
+            <includes>
+                <include>install.properties</include>
+            </includes>
+            <fileMode>700</fileMode>
+        </fileSet>
+        <fileSet>
             <outputDirectory>/ews/webapp/WEB-INF/classes/conf.dist</outputDirectory>
             <directoryMode>0700</directoryMode>
             <directory>kms/config/kms-webapp</directory>


[ranger] 07/39: RANGER-1958 [HBase] Implement getUserPermissions API of AccessControlService.Interface to allow clients to access HBase permissions stored in Ranger (Ankit Singhal)

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit 6af25a7ce2797a8b470b728f232f23376107c8d9
Author: Ankit Singhal <an...@gmail.com>
AuthorDate: Thu Oct 4 16:24:06 2018 -0700

    RANGER-1958 [HBase] Implement getUserPermissions API of AccessControlService.Interface to allow clients to access HBase permissions stored in Ranger (Ankit Singhal)
    
    Signed-off-by: rmani <rm...@hortonworks.com>
---
 .../authorization/hbase/AuthorizationSession.java  |   3 +-
 .../hbase/RangerAuthorizationCoprocessor.java      | 122 ++++++++++++++++++++-
 .../hbase/HBaseRangerAuthorizationTest.java        |  71 +++++++++++-
 hbase-agent/src/test/resources/hbase-policies.json |  58 ++++++++++
 4 files changed, 250 insertions(+), 4 deletions(-)

diff --git a/hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/AuthorizationSession.java b/hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/AuthorizationSession.java
index cdaad00..74293fb 100644
--- a/hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/AuthorizationSession.java
+++ b/hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/AuthorizationSession.java
@@ -174,7 +174,8 @@ public class AuthorizationSession {
 				StringUtils.equals(_operation, "deleteNamespace") ||
 				StringUtils.equals(_operation, "modifyNamespace") ||
 				StringUtils.equals(_operation, "setUserNamespaceQuota") ||
-				StringUtils.equals(_operation, "setNamespaceQuota");
+				StringUtils.equals(_operation, "setNamespaceQuota") ||
+				StringUtils.equals(_operation, "getUserPermissionForNamespace");
 	}
 
 	AuthorizationSession buildRequest() {
diff --git a/hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/RangerAuthorizationCoprocessor.java b/hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/RangerAuthorizationCoprocessor.java
index d85339a..ddb6d9b 100644
--- a/hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/RangerAuthorizationCoprocessor.java
+++ b/hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/RangerAuthorizationCoprocessor.java
@@ -19,6 +19,7 @@
 package org.apache.ranger.authorization.hbase;
 import java.io.IOException;
 import java.net.InetAddress;
+import java.security.PrivilegedExceptionAction;
 import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.Collection;
@@ -29,6 +30,7 @@ import java.util.Iterator;
 import java.util.LinkedList;
 import java.util.List;
 import java.util.Map;
+import java.util.Map.Entry;
 import java.util.NavigableSet;
 import java.util.Set;
 
@@ -36,6 +38,7 @@ import org.apache.commons.collections.CollectionUtils;
 import org.apache.commons.logging.Log;
 import org.apache.commons.logging.LogFactory;
 import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.AuthUtil;
 import org.apache.hadoop.hbase.Cell;
 import org.apache.hadoop.hbase.CoprocessorEnvironment;
 import org.apache.hadoop.hbase.HColumnDescriptor;
@@ -83,6 +86,7 @@ import org.apache.hadoop.hbase.regionserver.StoreFile;
 import org.apache.hadoop.hbase.regionserver.wal.WALEdit;
 import org.apache.hadoop.hbase.security.AccessDeniedException;
 import org.apache.hadoop.hbase.security.User;
+import org.apache.hadoop.hbase.security.access.AccessControlLists;
 import org.apache.hadoop.hbase.security.access.Permission;
 import org.apache.hadoop.hbase.security.access.Permission.Action;
 import org.apache.hadoop.hbase.security.access.RangerAccessControlLists;
@@ -97,13 +101,19 @@ import org.apache.ranger.authorization.hadoop.constants.RangerHadoopConstants;
 import org.apache.ranger.authorization.utils.StringUtil;
 import org.apache.ranger.plugin.audit.RangerDefaultAuditHandler;
 import org.apache.ranger.plugin.policyengine.RangerAccessRequest;
+import org.apache.ranger.plugin.policyengine.RangerAccessRequestImpl;
+import org.apache.ranger.plugin.policyengine.RangerAccessResourceImpl;
 import org.apache.ranger.plugin.policyengine.RangerAccessResultProcessor;
+import org.apache.ranger.plugin.policyengine.RangerResourceACLs;
+import org.apache.ranger.plugin.policyengine.RangerResourceACLs.AccessResult;
+import org.apache.ranger.plugin.policyevaluator.RangerPolicyEvaluator;
 import org.apache.ranger.plugin.service.RangerBasePlugin;
 import org.apache.ranger.plugin.util.GrantRevokeRequest;
 
 import com.google.common.base.Objects;
 import com.google.common.collect.Lists;
 import com.google.common.collect.MapMaker;
+import com.google.common.collect.Sets;
 import com.google.protobuf.RpcCallback;
 import com.google.protobuf.RpcController;
 import com.google.protobuf.Service;
@@ -1272,8 +1282,116 @@ public class RangerAuthorizationCoprocessor extends RangerAuthorizationCoprocess
 	}
 
 	@Override
-	public void getUserPermissions(RpcController controller, AccessControlProtos.GetUserPermissionsRequest request, RpcCallback<AccessControlProtos.GetUserPermissionsResponse> done) {
-		LOG.debug("getUserPermissions(): ");
+	public void getUserPermissions(RpcController controller, AccessControlProtos.GetUserPermissionsRequest request,
+			RpcCallback<AccessControlProtos.GetUserPermissionsResponse> done) {
+		AccessControlProtos.GetUserPermissionsResponse response = null;
+		try {
+			String operation = "userPermissions";
+			final RangerAccessResourceImpl resource = new RangerAccessResourceImpl();
+			User user = getActiveUser();
+			Set<String> groups = _userUtils.getUserGroups(user);
+			if (groups.isEmpty() && user.getUGI() != null) {
+				String[] groupArray = user.getUGI().getGroupNames();
+				if (groupArray != null) {
+					groups = Sets.newHashSet(groupArray);
+				}
+			}
+			RangerAccessRequestImpl rangerAccessrequest = new RangerAccessRequestImpl(resource, null,
+					_userUtils.getUserAsString(user), groups);
+			rangerAccessrequest.setAction(operation);
+			rangerAccessrequest.setClientIPAddress(getRemoteAddress());
+			rangerAccessrequest.setResourceMatchingScope(RangerAccessRequest.ResourceMatchingScope.SELF);
+			rangerAccessrequest.setClusterName(hbasePlugin.getClusterName());
+			List<UserPermission> perms = null;
+			if (request.getType() == AccessControlProtos.Permission.Type.Table) {
+				final TableName table = request.hasTableName() ? ProtobufUtil.toTableName(request.getTableName())
+						: null;
+				requirePermission(operation, table.getName(), Action.ADMIN);
+				resource.setValue(RangerHBaseResource.KEY_TABLE, table.getNameAsString());
+				perms = User.runAsLoginUser(new PrivilegedExceptionAction<List<UserPermission>>() {
+					@Override
+					public List<UserPermission> run() throws Exception {
+						return getUserPrermissions(
+								hbasePlugin.getCurrentRangerAuthContext().getResourceACLs(rangerAccessrequest),
+								table.getNameAsString(), false);
+					}
+				});
+			} else if (request.getType() == AccessControlProtos.Permission.Type.Namespace) {
+				final String namespace = request.getNamespaceName().toStringUtf8();
+				requireGlobalPermission("getUserPermissionForNamespace", namespace, Action.ADMIN);
+				resource.setValue(RangerHBaseResource.KEY_TABLE, namespace + RangerHBaseResource.NAMESPACE_SEPARATOR);
+				rangerAccessrequest.setRequestData(namespace);
+				perms = User.runAsLoginUser(new PrivilegedExceptionAction<List<UserPermission>>() {
+					@Override
+					public List<UserPermission> run() throws Exception {
+						return getUserPrermissions(
+								hbasePlugin.getCurrentRangerAuthContext().getResourceACLs(rangerAccessrequest),
+								namespace, true);
+					}
+				});
+			} else {
+				requirePermission("userPermissions", Action.ADMIN);
+				perms = User.runAsLoginUser(new PrivilegedExceptionAction<List<UserPermission>>() {
+					@Override
+					public List<UserPermission> run() throws Exception {
+						return getUserPrermissions(
+								hbasePlugin.getCurrentRangerAuthContext().getResourceACLs(rangerAccessrequest), null,
+								false);
+					}
+				});
+				if (_userUtils.isSuperUser(user)) {
+					perms.add(new UserPermission(Bytes.toBytes(_userUtils.getUserAsString(user)),
+							AccessControlLists.ACL_TABLE_NAME, null, Action.values()));
+				}
+			}
+			response = ResponseConverter.buildGetUserPermissionsResponse(perms);
+		} catch (IOException ioe) {
+			// pass exception back up
+			ResponseConverter.setControllerException(controller, ioe);
+		}
+		done.run(response);
+	}
+
+	private List<UserPermission> getUserPrermissions(RangerResourceACLs rangerResourceACLs, String resource,
+			boolean isNamespace) {
+		List<UserPermission> userPermissions = new ArrayList<UserPermission>();
+		Action[] hbaseActions = Action.values();
+		List<String> hbaseActionsList = new ArrayList<String>();
+		for (Action action : hbaseActions) {
+			hbaseActionsList.add(action.name());
+		}
+		addPermission(rangerResourceACLs.getUserACLs(), isNamespace, hbaseActionsList, userPermissions, resource,
+				false);
+		addPermission(rangerResourceACLs.getGroupACLs(), isNamespace, hbaseActionsList, userPermissions, resource,
+				true);
+		return userPermissions;
+	}
+
+	private void addPermission(Map<String, Map<String, AccessResult>> acls, boolean isNamespace,
+			List<String> hbaseActionsList, List<UserPermission> userPermissions, String resource, boolean isGroup) {
+		for (Entry<String, Map<String, AccessResult>> userAcls : acls.entrySet()) {
+			String user = !isGroup ? userAcls.getKey() : AuthUtil.toGroupEntry(userAcls.getKey());
+			List<Action> allowedPermissions = new ArrayList<Action>();
+			for (Entry<String, AccessResult> permissionAccess : userAcls.getValue().entrySet()) {
+				String permission = permissionAccess.getKey().toUpperCase();
+				if (hbaseActionsList.contains(permission)
+						&& permissionAccess.getValue().getResult() == RangerPolicyEvaluator.ACCESS_ALLOWED) {
+					allowedPermissions.add(Action.valueOf(permission));
+				}
+
+			}
+			if (!allowedPermissions.isEmpty()) {
+				UserPermission up = null;
+				if (isNamespace) {
+					up = new UserPermission(Bytes.toBytes(user), resource,
+							allowedPermissions.toArray(new Action[allowedPermissions.size()]));
+				} else {
+					up = new UserPermission(Bytes.toBytes(user), TableName.valueOf(resource), null, null,
+							allowedPermissions.toArray(new Action[allowedPermissions.size()]));
+				}
+				userPermissions.add(up);
+			}
+		}
 	}
 
 	@Override
diff --git a/hbase-agent/src/test/java/org/apache/ranger/authorization/hbase/HBaseRangerAuthorizationTest.java b/hbase-agent/src/test/java/org/apache/ranger/authorization/hbase/HBaseRangerAuthorizationTest.java
index 3840885..f1cd893 100644
--- a/hbase-agent/src/test/java/org/apache/ranger/authorization/hbase/HBaseRangerAuthorizationTest.java
+++ b/hbase-agent/src/test/java/org/apache/ranger/authorization/hbase/HBaseRangerAuthorizationTest.java
@@ -42,6 +42,9 @@ import org.apache.hadoop.hbase.client.Put;
 import org.apache.hadoop.hbase.client.Result;
 import org.apache.hadoop.hbase.client.Table;
 import org.apache.hadoop.hbase.protobuf.generated.HBaseProtos;
+import org.apache.hadoop.hbase.security.access.AccessControlClient;
+import org.apache.hadoop.hbase.security.access.Permission;
+import org.apache.hadoop.hbase.security.access.UserPermission;
 import org.apache.hadoop.hbase.util.Bytes;
 import org.apache.hadoop.security.UserGroupInformation;
 import org.junit.Assert;
@@ -116,6 +119,15 @@ public class HBaseRangerAuthorizationTest {
             admin.createTable(tableDescriptor);
         }
 
+		if (!admin.tableExists(TableName.valueOf("default:temp5"))) {
+			HTableDescriptor tableDescriptor = new HTableDescriptor(TableName.valueOf("default:temp5"));
+
+			// Adding column families to table descriptor
+			tableDescriptor.addFamily(new HColumnDescriptor("colfam1"));
+
+			admin.createTable(tableDescriptor);
+		}
+
         // Add a new row
         Put put = new Put(Bytes.toBytes("row1"));
         put.addColumn(Bytes.toBytes("colfam1"), Bytes.toBytes("col1"), Bytes.toBytes("val1"));
@@ -174,7 +186,7 @@ public class HBaseRangerAuthorizationTest {
         for (HTableDescriptor desc : tableDescriptors) {
             LOG.info("Found table:[" + desc.getTableName().getNameAsString() + "]");
         }
-        Assert.assertEquals(2, tableDescriptors.length);
+        Assert.assertEquals(3, tableDescriptors.length);
 
         conn.close();
     }
@@ -961,6 +973,63 @@ public class HBaseRangerAuthorizationTest {
         conn.close();
     }
 
+	@Test
+	public void testGetUserPermission() throws Throwable {
+		final Configuration conf = HBaseConfiguration.create();
+		conf.set("hbase.zookeeper.quorum", "localhost");
+		conf.set("hbase.zookeeper.property.clientPort", "" + port);
+		conf.set("zookeeper.znode.parent", "/hbase-unsecure");
+		String user = "IT";
+		UserGroupInformation ugi = UserGroupInformation.createUserForTesting(user, new String[] { "IT" });
+		ugi.doAs(new PrivilegedExceptionAction<Void>() {
+			public Void run() throws Exception {
+				try (Connection conn = ConnectionFactory.createConnection(conf)) {
+					AccessControlClient.getUserPermissions(conn, "temp");
+					Assert.fail();
+				} catch (Throwable e) {
+					// expected
+				}
+				return null;
+			}
+
+		});
+
+		user = "QA";
+		ugi = UserGroupInformation.createUserForTesting(user, new String[] { "QA" });
+		ugi.doAs(new PrivilegedExceptionAction<Void>() {
+			public Void run() throws Exception {
+				List<UserPermission> userPermissions;
+				try (Connection conn = ConnectionFactory.createConnection(conf)) {
+					userPermissions = AccessControlClient.getUserPermissions(conn, "@test_namespace");
+				} catch (Throwable e) {
+					throw new Exception(e);
+				}
+				boolean found = false;
+				for (UserPermission namespacePermission : userPermissions) {
+					if (namespacePermission.hasNamespace()) {
+						found = Bytes.equals(namespacePermission.getUser(), Bytes.toBytes("@QA"));
+						if (found) {
+							break;
+						}
+					}
+				}
+				Assert.assertTrue("QA is not found", found);
+				return null;
+			}
+		});
+
+		List<UserPermission> userPermissions;
+		try (Connection conn = ConnectionFactory.createConnection(conf)) {
+			userPermissions = AccessControlClient.getUserPermissions(conn, "temp5");
+		} catch (Throwable e) {
+			throw new Exception(e);
+		}
+		UserPermission userPermission = new UserPermission(Bytes.toBytes("@IT"), TableName.valueOf("temp5"), null,
+				Permission.Action.READ, Permission.Action.WRITE);
+		Assert.assertTrue("@IT permission should be there", userPermissions.contains(userPermission));
+
+	}
+
     private static int getFreePort() throws IOException {
         ServerSocket serverSocket = new ServerSocket(0);
         int port = serverSocket.getLocalPort();
diff --git a/hbase-agent/src/test/resources/hbase-policies.json b/hbase-agent/src/test/resources/hbase-policies.json
index b7b44c9..6213a0e 100644
--- a/hbase-agent/src/test/resources/hbase-policies.json
+++ b/hbase-agent/src/test/resources/hbase-policies.json
@@ -132,6 +132,64 @@
     },
     {
       "service": "cl1_hbase",
+      "name": "TempPolicy",
+      "policyType": 0,
+      "description": "",
+      "isAuditEnabled": true,
+      "resources": {
+        "column-family": {
+          "values": [
+            "*"
+          ],
+          "isExcludes": false,
+          "isRecursive": false
+        },
+        "column": {
+          "values": [
+            "*"
+          ],
+          "isExcludes": false,
+          "isRecursive": false
+        },
+        "table": {
+          "values": [
+            "temp5"
+          ],
+          "isExcludes": false,
+          "isRecursive": false
+        }
+      },
+      "policyItems": [
+        {
+          "accesses": [
+            {
+              "type": "read",
+              "isAllowed": true
+            },
+            {
+              "type": "write",
+              "isAllowed": true
+            }
+          ],
+          "users": [],
+          "groups": [
+            "IT"
+          ],
+          "conditions": [],
+          "delegateAdmin": false
+        }
+      ],
+      "denyPolicyItems": [],
+      "allowExceptions": [],
+      "denyExceptions": [],
+      "dataMaskPolicyItems": [],
+      "rowFilterPolicyItems": [],
+      "id": 33,
+      "isEnabled": true,
+      "version": 1
+    },
+    {
+      "service": "cl1_hbase",
       "name": "HBASETest-3-namespace",
       "description": "Default Policy for Service: HBASETest for namespace test_namespace",
       "isAuditEnabled": true,


[ranger] 26/39: RANGER-2288: Sqoop repository config missing 'Common Name for Certificate'

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit 6ec3f991acc9c796354439717904b7985f39215f
Author: zhangqiang2 <zh...@zte.com.cn>
AuthorDate: Fri Nov 16 15:10:44 2018 +0800

    RANGER-2288: Sqoop repository config missing 'Common Name for Certificate'
    
    Signed-off-by: peng.jianhua <pe...@zte.com.cn>
---
 .../resources/service-defs/ranger-servicedef-sqoop.json     | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json b/agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json
index 902a0b8..8cff9ab 100644
--- a/agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json
+++ b/agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json
@@ -103,7 +103,18 @@
 			"validationMessage": "",
 			"uiHint":"{\"TextFieldWithIcon\":true, \"info\": \"eg. 'http://&lt;ipaddr&gt;:12000'\"}",
 			"label": "Sqoop URL"
-		}
+		},
+
+        {
+            "itemId": 3,
+            "name": "commonNameForCertificate",
+            "type": "string",
+            "mandatory": false,
+            "validationRegEx":"",
+            "validationMessage": "",
+            "uiHint":"",
+            "label": "Common Name for Certificate"
+        }
 
 	],
 	"options": { "enableDenyAndExceptionsInPolicies": "false" },


[ranger] 35/39: RANGER-2163:Spelling error in the PatchPersmissionModel_J10003.java

Posted by me...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

mehul pushed a commit to branch ranger-1.1
in repository https://gitbox.apache.org/repos/asf/ranger.git

commit 9d07e834029e2f409e3a7321112a6bac5ab480a6
Author: zhangqiang2 <zh...@zte.com.cn>
AuthorDate: Thu Jul 26 16:14:06 2018 +0800

    RANGER-2163:Spelling error in the PatchPersmissionModel_J10003.java
    
    Signed-off-by: peng.jianhua <pe...@zte.com.cn>
---
 ...smissionModel_J10003.java => PatchPermissionModel_J10003.java} | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/security-admin/src/main/java/org/apache/ranger/patch/PatchPersmissionModel_J10003.java b/security-admin/src/main/java/org/apache/ranger/patch/PatchPermissionModel_J10003.java
similarity index 96%
rename from security-admin/src/main/java/org/apache/ranger/patch/PatchPersmissionModel_J10003.java
rename to security-admin/src/main/java/org/apache/ranger/patch/PatchPermissionModel_J10003.java
index 89bfd9f..4a38d0a 100644
--- a/security-admin/src/main/java/org/apache/ranger/patch/PatchPersmissionModel_J10003.java
+++ b/security-admin/src/main/java/org/apache/ranger/patch/PatchPermissionModel_J10003.java
@@ -39,9 +39,9 @@ import org.springframework.stereotype.Component;
 import org.springframework.util.CollectionUtils;
 
 @Component
-public class PatchPersmissionModel_J10003 extends BaseLoader {
+public class PatchPermissionModel_J10003 extends BaseLoader {
 	private static final Logger logger = Logger
-			.getLogger(PatchPersmissionModel_J10003.class);
+			.getLogger(PatchPermissionModel_J10003.class);
 
 	@Autowired
 	XUserMgr xUserMgr;
@@ -65,8 +65,8 @@ public class PatchPersmissionModel_J10003 extends BaseLoader {
 					usersListFileName=args[0];
 				}
 			}
-			PatchPersmissionModel_J10003 loader = (PatchPersmissionModel_J10003) CLIUtil
-					.getBean(PatchPersmissionModel_J10003.class);
+			PatchPermissionModel_J10003 loader = (PatchPermissionModel_J10003) CLIUtil
+					.getBean(PatchPermissionModel_J10003.class);
 
 			loader.init();
 			while (loader.isMoreToProcess()) {