You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ranger.apache.org by dh...@apache.org on 2021/09/09 09:38:15 UTC
[ranger] reference refs/for/master updated (2f569ee -> 7e80592)
This is an automated email from the ASF dual-hosted git repository.
dhavalshah9131 pushed a change to reference refs/for/master
in repository https://gitbox.apache.org/repos/asf/ranger.git.
from 2f569ee RANGER : 3328 - RANGER-KMS : code improvement
add 5b075e6 RANGER-3343: Ranger policy cache is incorrect in some scenario
add db2bd7c RANGER-3329: Request for _any access-type is denied only when on all access-types are denied
add 280d0a6 RANGER-3332: script evaluator updated to provide JavaScript style access to details of request/user/groups/tags
add 9c722cd RANGER-3342 : Need to make the Ranger embedded server work directory configurable
add 18ecd31 RANGER-3344:Ranger Admin fails to start with ava.lang.NoClassDefFoundError: org/apache/htrace/core/Tracer
add 83e3b19 RANGER-3342 : Addendum fix Need to make the Ranger embedded server work directory configurable
add dadf921 RANGER-3351: Incorrect hive query displayed for grant and revoke role command
add 8248039 RANGER-3345 : Default Ranger policy for KMS should include 'om' user for Ozone bucket level encryption to work
add b2406be RANGER-3349 : Handling multiple grant role command for same user
add 203cd27 RANGER-3308: Create python script to test stability of policy CRUD
add 16c5145 RANGER-3347: Add default policy for hbase user in hdfs services
add a7ba77e RANGER-3024: Improve response time and refactor code for GET API /service/xusers/lookup/users
add cfc0330 RANGER-3259 : [Ranger Audit Filter] Ranger role is allowed to delete, even if its used in audit filters
add c9003ff RANGER-3360: Best Practice: Use updated policy object after pruning the policy object
add 91545f7 RANGER-3329: Request for _any access-type is denied only when on all access-types are denied - follow-up
add def49c7 RANNGER-3355 : Update the current logging mechanism to use custom log4j conf
add 9115a20 RANGER-3357:Ranger HivePlugin Authorization for a new Hive operation
add 1cd7da9 RANGER-3353:Show roles is not listing all roles
add e9af2c1 RANGER-3368:Ranger HiveAuthorizer improvements to handle uncharted hive commands
add bed66ec RANGER-3370: updated Python client to handle 404 HTTPStatus code
add bf2d163 RANGER-3366: Cluster type is missed in copy constructor of RangerAccessRequestImpl
add dfc018b RANGER-3362 : UI Improvements.
add 10d0a83 RANGER-3367: [Intermittent] Ranger Admin perf logs are not getting logged after Spring Security upgrade
add 8f177b0 RANGER-3361 : Improve error message while deleting users and groups associated with role
add 6030613 RANGER-3371: Update algorithm to build Ranger policy-database object from Ranger policy-view object
add 3e381db RANGER-3348: Add user & group delete functionality in Apache Ranger Python APIs
add 1dee2b3 RANGER-3334:Enhance Ranger admin REST Client to use cookie for policy, tag and role download
add 224e5fb Introduce new column syncSource for Ranger UI
add f91d702 RANGER-3374: Creating new DB transaction for add/update of user group mapping from Ranger Usersync
add e1ec224 RANGER-3348: Add user & group delete functionality in Apache Ranger Python APIs - follow-up
add 4b97613 RANGER-3376: Add policy_guid column in x_policy_change_log table
add f8005de RANGER-3285: Addendum patch to fix procedure name in mysql patch
add c2a9833 RANGER-3372 : ISSUE IN POLICIES SEARCH ON REPORT PAGE WITH USER HAVING MORE THAN ONE UNIX GROUPS
add b0e970f RANGER-3377: HDFS plugin performance improvement - RangerHdfsResource.getAsString()
add fbc08b3 Revert "RANGER-3377: HDFS plugin performance improvement - RangerHdfsResource.getAsString()"
add 636dcd7 RANGER-3378: HDFS plugin performance improvement - RangerHdfsResource.getAsString()
add 79f6cde RANGER-3377: HDFS plugin performance improvement - conditionally ignore deny and exception conditions
add fcfed20 RANGER-3293 : Show user source details on user tab in ranger UI.
add 93a15b8 RANGER-3285: Addendum patch to fix a syntax in oracle patch
add d0b2ab4 RANGER-3363: Added support in ranger admin for handling session timeout requests with knox proxy
add f4d18bf RANGER-3385: Duplicate SQL prefix should not be allowed
add 8cfcecb RANGER-3396: fixed incorrect class name in RangerPolicyItemRowFilterInfo.toString()
add fe27e0b RANGER-3397: Update ACL computation to (optionally) expand Ranger Roles to users and groups and include chained-plugins in ACL computation
add 00d660c RANGER-3398: Duplicate JAVA patch suffix should not be allowed
add 43e3e83 RANGER-3402: updated getResourceACLs() to avoid references to collections in RangerPolicy
add e6ef861 RANGER-3358 : Upgrade Tomcat to 8.5.69
add 92fdf20 RANGER-3350: Ranger HivePluginAuthorizer SHOW CURRENT ROLES not fetching the role set in current hive beeline session
add 3b0a9c8 RANGER-3397: Update ACL computation to (optionally) expand Ranger Roles to users and groups and include chained-plugins in ACL computation - Part 2
new 7e80592 RANGER-3339 : Make Ranger Solr audit collection storage configurable
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../ranger/admin/client/RangerAdminRESTClient.java | 756 ++++++++++++-----
.../ranger/authorization/utils/JsonUtils.java | 27 +-
.../RangerScriptConditionEvaluator.java | 26 +
.../RangerScriptExecutionContext.java | 113 +++
.../RangerAbstractContextEnricher.java | 4 +
.../contextenricher/RangerAdminTagRetriever.java | 7 +-
.../RangerAdminUserStoreRetriever.java | 7 +-
.../plugin/contextenricher/RangerTagEnricher.java | 1 +
.../plugin/contextenricher/RangerTagRetriever.java | 10 +
.../contextenricher/RangerUserStoreEnricher.java | 1 +
.../contextenricher/RangerUserStoreRetriever.java | 10 +
.../apache/ranger/plugin/model/RangerPolicy.java | 12 +-
.../policyengine/RangerAccessRequestImpl.java | 20 +
.../plugin/policyengine/RangerPluginContext.java | 58 ++
.../plugin/policyengine/RangerPolicyEngine.java | 2 +
.../policyengine/RangerPolicyEngineImpl.java | 117 ++-
.../policyengine/RangerPolicyEngineOptions.java | 11 +-
.../RangerAbstractPolicyEvaluator.java | 64 +-
.../RangerDefaultPolicyEvaluator.java | 226 +++--
.../policyevaluator/RangerPolicyEvaluator.java | 8 +-
.../ranger/plugin/service/RangerBasePlugin.java | 133 ++-
.../ranger/plugin/service/RangerChainedPlugin.java | 7 +
.../apache/ranger/plugin/util/PolicyRefresher.java | 5 +-
.../plugin/util/RangerAccessRequestUtil.java | 9 +
.../ranger/plugin/util/RangerCommonConstants.java | 26 +
.../ranger/plugin/util/RangerPolicyDeltaUtil.java | 2 +-
.../ranger/plugin/util/RangerRESTClient.java | 26 +
.../apache/ranger/plugin/util/RangerRolesUtil.java | 64 +-
.../RangerCustomConditionMatcherTest.java | 106 ++-
.../ranger/plugin/policyengine/TestPolicyACLs.java | 10 +-
.../policyengine/test_aclprovider_hdfs.json | 131 +++
.../test_policyengine_descendant_tags.json | 8 +-
.../policyengine/test_policyengine_hive.json | 2 +-
.../policyengine/test_policyengine_tag_hive.json | 14 +
...t_policyengine_tag_hive_for_show_databases.json | 10 +-
distro/src/main/assembly/admin-web.xml | 9 +
embeddedwebserver/scripts/ranger-admin-services.sh | 8 +-
.../ranger/server/tomcat/EmbeddedServer.java | 16 +-
.../server/tomcat/SolrCollectionBootstrapper.java | 75 +-
.../authorization/hadoop/RangerHdfsAuthorizer.java | 13 +
.../ranger/services/hdfs/RangerServiceHdfs.java | 116 ++-
.../aclprovider/test_aclprovider_default.json | 142 ++++
.../hive/authorizer/RangerHiveAuditHandler.java | 25 +
.../hive/authorizer/RangerHiveAuthorizer.java | 136 ++-
.../python/apache_ranger/client/ranger_client.py | 248 +++---
intg/src/main/python/apache_ranger/utils.py | 1 +
.../admin/client/RangerAdminJersey2RESTClient.java | 915 ++++++++++++++++-----
.../ranger/services/kms/RangerServiceKMS.java | 13 +-
pom.xml | 2 +-
.../optimized/current/ranger_core_db_mysql.sql | 6 +
...Source-col-in-x_user-x_portal_user-x_group.sql} | 38 +-
... 056-add-policyguid-in-x_policy_change_log.sql} | 10 +-
.../optimized/current/ranger_core_db_oracle.sql | 6 +
...Source-col-in-x_user-x_portal_user-x_group.sql} | 16 +-
... 056-add-policyguid-in-x_policy_change_log.sql} | 4 +-
.../optimized/current/ranger_core_db_postgres.sql | 6 +
...Source-col-in-x_user-x_portal_user-x_group.sql} | 22 +-
... 056-add-policyguid-in-x_policy_change_log.sql} | 8 +-
.../current/ranger_core_db_sqlanywhere.sql | 10 +-
...Source-col-in-x_user-x_portal_user-x_group.sql} | 15 +-
... 056-add-policyguid-in-x_policy_change_log.sql} | 7 +-
.../optimized/current/ranger_core_db_sqlserver.sql | 6 +
...Source-col-in-x_user-x_portal_user-x_group.sql} | 16 +-
... 056-add-policyguid-in-x_policy_change_log.sql} | 7 +-
security-admin/pom.xml | 112 +++
security-admin/scripts/db_setup.py | 43 +-
security-admin/scripts/install.properties | 2 +
security-admin/scripts/setup.sh | 16 +
.../java/org/apache/ranger/biz/RoleDBStore.java | 8 +
.../java/org/apache/ranger/biz/ServiceDBStore.java | 238 +++++-
.../main/java/org/apache/ranger/biz/UserMgr.java | 3 +-
.../main/java/org/apache/ranger/biz/XUserMgr.java | 92 ++-
.../ranger/common/RangerServicePoliciesCache.java | 6 +-
.../RangerTransactionSynchronizationAdapter.java | 27 +-
.../org/apache/ranger/db/XXPolicyChangeLogDao.java | 15 +-
.../apache/ranger/db/XXServiceConfigMapDao.java | 13 +
.../java/org/apache/ranger/entity/XXGroup.java | 24 +
.../apache/ranger/entity/XXPolicyChangeLog.java | 15 +-
.../org/apache/ranger/entity/XXPortalUser.java | 25 +
.../main/java/org/apache/ranger/entity/XXUser.java | 25 +
.../main/java/org/apache/ranger/rest/RoleREST.java | 46 +-
.../main/java/org/apache/ranger/rest/UserREST.java | 7 +
.../java/org/apache/ranger/rest/XUserREST.java | 9 +-
.../RangerAuthenticationEntryPoint.java | 4 +-
.../web/filter/RangerKRBAuthenticationFilter.java | 44 +-
.../web/filter/RangerSSOAuthenticationFilter.java | 96 +--
.../ranger/service/RangerPolicyServiceBase.java | 28 +-
.../org/apache/ranger/service/XGroupService.java | 4 +-
.../apache/ranger/service/XGroupServiceBase.java | 8 +-
.../apache/ranger/service/XGroupUserService.java | 74 +-
.../ranger/service/XPortalUserServiceBase.java | 2 +
.../org/apache/ranger/service/XUserService.java | 21 +
.../apache/ranger/service/XUserServiceBase.java | 9 +-
.../main/java/org/apache/ranger/util/RestUtil.java | 85 ++
.../main/java/org/apache/ranger/view/VXGroup.java | 19 +
.../java/org/apache/ranger/view/VXPortalUser.java | 37 +
.../main/java/org/apache/ranger/view/VXUser.java | 17 +
.../main/resources/META-INF/jpa_named_queries.xml | 4 +
.../conf.dist/ranger-admin-default-site.xml | 4 +
.../main/resources/conf.dist/ranger-admin-site.xml | 8 +
.../src/main/webapp/WEB-INF/log4j.properties | 16 +
.../src/main/webapp/scripts/utils/XAViewUtils.js | 9 +
.../scripts/views/policies/PermissionList.js | 11 +-
.../main/webapp/scripts/views/users/GroupForm.js | 10 +
.../main/webapp/scripts/views/users/UserForm.js | 34 +-
.../webapp/scripts/views/users/UserTableLayout.js | 120 ++-
.../webapp/templates/users/GroupForm_tmpl.html | 25 +
.../main/webapp/templates/users/UserForm_tmpl.html | 29 +
.../java/org/apache/ranger/biz/TestXUserMgr.java | 20 +
.../stability-tests/ranger-policy/app.conf | 10 +-
.../stability-tests/ranger-policy/start.sh | 98 +++
.../ranger-policy/test-hdfs-policy.py | 245 ++++++
.../apache/ranger/ugsyncutil/model/XGroupInfo.java | 5 +
.../apache/ranger/ugsyncutil/model/XUserInfo.java | 10 +
.../process/PolicyMgrUserGroupBuilder.java | 5 +-
115 files changed, 4631 insertions(+), 1135 deletions(-)
create mode 100644 agents-common/src/test/resources/policyengine/test_aclprovider_hdfs.json
create mode 100644 hdfs-agent/src/test/resources/aclprovider/test_aclprovider_default.json
copy security-admin/db/mysql/patches/{045-add-displayName-col-in-x_service_def_and_x_service.sql => 055-add-syncSource-col-in-x_user-x_portal_user-x_group.sql} (53%)
copy security-admin/db/mysql/patches/{054-add-version-in-x_service_version_info.sql => 056-add-policyguid-in-x_policy_change_log.sql} (71%)
copy security-admin/db/oracle/patches/{044-add-role-version-in-serviceVersionInfo.sql => 055-add-syncSource-col-in-x_user-x_portal_user-x_group.sql} (62%)
copy security-admin/db/oracle/patches/{054-add-version-in-x_service_version_info.sql => 056-add-policyguid-in-x_policy_change_log.sql} (84%)
copy security-admin/db/postgres/patches/{044-add-role-version-in-serviceVersionInfo.sql => 055-add-syncSource-col-in-x_user-x_portal_user-x_group.sql} (56%)
copy security-admin/db/postgres/patches/{054-add-version-in-x_service_version_info.sql => 056-add-policyguid-in-x_policy_change_log.sql} (78%)
copy security-admin/db/sqlanywhere/patches/{032-add-options-to-policy-and-tag-for-time-based-processing.sql => 055-add-syncSource-col-in-x_user-x_portal_user-x_group.sql} (60%)
copy security-admin/db/sqlanywhere/patches/{039-add-column-version-in-x_policy_export_audit.sql => 056-add-policyguid-in-x_policy_change_log.sql} (85%)
copy security-admin/db/sqlserver/patches/{032-add-options-to-policy-and-tag-for-time-based-processing.sql => 055-add-syncSource-col-in-x_user-x_portal_user-x_group.sql} (71%)
copy security-admin/db/sqlserver/patches/{021-update-tag-for-owner.sql => 056-add-policyguid-in-x_policy_change_log.sql} (83%)
copy embeddedwebserver/scripts/stop-ranger-admin.sh => security-admin/src/test/resources/stability-tests/ranger-policy/app.conf (57%)
mode change 100755 => 100644
create mode 100755 security-admin/src/test/resources/stability-tests/ranger-policy/start.sh
create mode 100755 security-admin/src/test/resources/stability-tests/ranger-policy/test-hdfs-policy.py
[ranger] 01/01: RANGER-3339 : Make Ranger Solr audit collection
storage configurable
Posted by dh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dhavalshah9131 pushed a commit to reference refs/for/master
in repository https://gitbox.apache.org/repos/asf/ranger.git
commit 7e80592481306bb0711f7a7544b2c6c64cbebadf
Author: mateenmansoori <ma...@gmail.com>
AuthorDate: Tue Aug 17 19:54:38 2021 +0530
RANGER-3339 : Make Ranger Solr audit collection storage configurable
---
embeddedwebserver/scripts/ranger-admin-services.sh | 2 +-
.../server/tomcat/SolrCollectionBootstrapper.java | 75 ++++++++++++++--------
security-admin/scripts/install.properties | 1 +
security-admin/scripts/setup.sh | 8 +++
.../main/resources/conf.dist/ranger-admin-site.xml | 4 ++
5 files changed, 61 insertions(+), 29 deletions(-)
diff --git a/embeddedwebserver/scripts/ranger-admin-services.sh b/embeddedwebserver/scripts/ranger-admin-services.sh
index d7cabbc..bced4ad 100755
--- a/embeddedwebserver/scripts/ranger-admin-services.sh
+++ b/embeddedwebserver/scripts/ranger-admin-services.sh
@@ -91,7 +91,7 @@ fi
SERVER_NAME=rangeradmin
start() {
SLEEP_TIME_AFTER_START=5
- nohup java -Dproc_rangeradmin ${JAVA_OPTS} -Dlog4j.configuration=file:${RANGER_ADMIN_LOG4J_CONF_FILE} -Duser=${USER} -Dhostname=${HOSTNAME} ${DB_SSL_PARAM} -Dservername=${SERVER_NAME} -Dlogdir=${RANGER_ADMIN_LOG_DIR} -Dcatalina.base=${XAPOLICYMGR_EWS_DIR} -cp "${XAPOLICYMGR_EWS_DIR}/webapp/WEB-INF/classes/conf:${XAPOLICYMGR_EWS_DIR}/lib/*:${RANGER_JAAS_LIB_DIR}/*:${RANGER_JAAS_CONF_DIR}:${JAVA_HOME}/lib/*:${RANGER_HADOOP_CONF_DIR}/*:$CLASSPATH" org.apache.ranger.server.tomcat.Embedded [...]
+ nohup java -Dproc_rangeradmin ${JAVA_OPTS} -Dlog4j.configuration=file:${RANGER_ADMIN_LOG4J_CONF_FILE} -Duser=${USER} -Dhostname=${HOSTNAME} ${DB_SSL_PARAM} -Dservername=${SERVER_NAME} -Dlogdir=${RANGER_ADMIN_LOG_DIR} -Dcatalina.base=${XAPOLICYMGR_EWS_DIR} -cp "${XAPOLICYMGR_EWS_DIR}/webapp/WEB-INF/classes/conf:${XAPOLICYMGR_EWS_DIR}/lib/*:${XAPOLICYMGR_EWS_DIR}/webapp/WEB-INF/lib/*:${RANGER_JAAS_LIB_DIR}/*:${RANGER_JAAS_CONF_DIR}:${JAVA_HOME}/lib/*:${RANGER_HADOOP_CONF_DIR}/*:$CLASSPA [...]
VALUE_OF_PID=$!
echo "Starting Apache Ranger Admin Service"
sleep $SLEEP_TIME_AFTER_START
diff --git a/embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrCollectionBootstrapper.java b/embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrCollectionBootstrapper.java
index d04d1c8..1596beb 100644
--- a/embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrCollectionBootstrapper.java
+++ b/embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrCollectionBootstrapper.java
@@ -35,6 +35,8 @@ import java.util.Optional;
import java.util.Set;
import java.util.logging.Level;
import java.util.logging.Logger;
+
+import org.apache.commons.lang.StringUtils;
import org.apache.hadoop.security.SecureClientLogin;
import org.apache.http.HttpEntity;
import org.apache.http.client.methods.HttpPost;
@@ -65,6 +67,7 @@ public class SolrCollectionBootstrapper extends Thread {
final static String SOLR_ZK_HOSTS = "ranger.audit.solr.zookeepers";
final static String SOLR_COLLECTION_NAME = "ranger.audit.solr.collection.name";
final static String SOLR_CONFIG_NAME = "ranger.audit.solr.config.name";
+ final static String CONFIG_SET_LOCATION = "ranger.audit.solr.configset.location";
final static String SOLR_NO_SHARDS = "ranger.audit.solr.no.shards";
final static String SOLR_MAX_SHARD_PER_NODE = "ranger.audit.solr.max.shards.per.node";
final static String SOLR_NO_REPLICA = "ranger.audit.solr.no.replica";
@@ -94,6 +97,7 @@ public class SolrCollectionBootstrapper extends Thread {
String nameRules;
String solr_collection_name;
String solr_config_name;
+ private String customConfigSetLocation;
Path path_for_cloud_mode;
int no_of_replicas;
int max_node_per_shards;
@@ -149,29 +153,26 @@ public class SolrCollectionBootstrapper extends Thread {
String basedir = new File(".").getCanonicalPath();
String solrFileDir = new File(basedir).getParent();
- path_for_cloud_mode = Paths.get(solrFileDir, "contrib",
- "solr_for_audit_setup", "conf");
- configSetFolder = path_for_cloud_mode.toFile();
+ this.customConfigSetLocation = EmbeddedServerUtil.getConfig(CONFIG_SET_LOCATION);
+ logger.info("Provided custom configSet location : " + this.customConfigSetLocation);
+ if (StringUtils.isNotEmpty(this.customConfigSetLocation)) {
+ this.configSetFolder = new File(this.customConfigSetLocation);
+ } else {
+ path_for_cloud_mode = Paths.get(solrFileDir, "contrib", "solr_for_audit_setup", "conf");
+ configSetFolder = path_for_cloud_mode.toFile();
+ }
String sslEnabledProp = EmbeddedServerUtil.getConfig(SSL_ENABLED_PARAM);
isSSLEnabled = ("true".equalsIgnoreCase(sslEnabledProp));
}
public void run() {
logger.info("Started run method");
-
- String zkHosts = "";
- List<String> zookeeperHosts = null;
- if (EmbeddedServerUtil.getConfig(SOLR_ZK_HOSTS) != null
- && !StringUtil.isEmpty(EmbeddedServerUtil.getConfig(SOLR_ZK_HOSTS))) {
- zkHosts = EmbeddedServerUtil.getConfig(SOLR_ZK_HOSTS).trim();
- zookeeperHosts = new ArrayList<String>(Arrays.asList(zkHosts
- .split(",")));
- }
+ List<String> zookeeperHosts = getZkHosts();
if (zookeeperHosts != null
&& !zookeeperHosts.isEmpty()
&& zookeeperHosts.stream().noneMatch(
h -> h.equalsIgnoreCase("none"))) {
- logger.info("Solr zkHosts=" + zkHosts + ", collectionName="
+ logger.info("Solr zkHosts=" + zookeeperHosts + ", collectionName="
+ solr_collection_name);
while (!is_completed && (max_retry == TRY_UNTIL_SUCCESS || retry_counter < max_retry)) {
try {
@@ -215,6 +216,8 @@ public class SolrCollectionBootstrapper extends Thread {
solrCloudClient = new CloudSolrClient.Builder(zookeeperHosts,
Optional.empty()).build();
solrCloudClient.setDefaultCollection(solr_collection_name);
+ solrCloudClient.connect();
+ zkClient = solrCloudClient.getZkStateReader().getZkClient();
solrClient = solrCloudClient;
solr_cloud_mode = true;
@@ -263,9 +266,6 @@ public class SolrCollectionBootstrapper extends Thread {
private boolean uploadConfiguration() {
try {
- solrCloudClient.connect();
- zkClient = solrCloudClient.getZkStateReader().getZkClient();
-
if (zkClient != null) {
ZkConfigManager zkConfigManager = new ZkConfigManager(zkClient);
@@ -274,12 +274,18 @@ public class SolrCollectionBootstrapper extends Thread {
try {
logger.info("Config does not exist with name " + solr_config_name);
String zipOfConfigs = null;
- String[] files = configSetFolder.list();
- for (String file : files) {
- if (file != null) {
- if (file.equals("solr_audit_conf.zip")) {
- zipOfConfigs = file;
- break;
+ if (this.configSetFolder.exists() && this.configSetFolder.isFile()) {
+ zipOfConfigs = this.configSetFolder.getAbsolutePath();
+ } else {
+ String[] files = this.configSetFolder.list();
+ if (files != null) {
+ for (String aFile : files) {
+ if (aFile != null) {
+ if (aFile.equals("solr_audit_conf.zip")) {
+ zipOfConfigs = this.configSetFolder + "/" + aFile;
+ break;
+ }
+ }
}
}
}
@@ -287,14 +293,10 @@ public class SolrCollectionBootstrapper extends Thread {
throw new FileNotFoundException(
"Could Not Find Configs Zip File : " + getConfigSetFolder());
}
- File file = new File(configSetFolder + "/" + zipOfConfigs);
+ File file = new File(zipOfConfigs);
byte[] arrByte = Files.readAllBytes(file.toPath());
ByteBuffer byteBuffer = ByteBuffer.wrap(arrByte);
- Set<String> nodes = solrCloudClient.getClusterStateProvider().getLiveNodes();
- String baseUrl = null;
- String[] nodeArr = nodes.toArray(new String[0]);
- /* getting nodes URL as 'solr_8983', so converting it to 'solr/9893' */
- baseUrl = nodeArr[0].replaceAll("_", "/");
+ String baseUrl = getBaseUrl();
String protocol = isSSLEnabled ? "https" : "http";
String uploadConfigsUrl = String.format("%s://%s/admin/configs?action=UPLOAD&name=%s", protocol,
baseUrl.toString(), solr_config_name);
@@ -447,4 +449,21 @@ public class SolrCollectionBootstrapper extends Thread {
return configSetFolder;
}
+ private static List<String> getZkHosts() {
+ String zkHosts = "";
+ List<String> zookeeperHosts = null;
+ if (!StringUtil.isEmpty(EmbeddedServerUtil.getConfig(SOLR_ZK_HOSTS))) {
+ zkHosts = EmbeddedServerUtil.getConfig(SOLR_ZK_HOSTS).trim();
+ zookeeperHosts = new ArrayList<String>(Arrays.asList(zkHosts.split(",")));
+ }
+ return zookeeperHosts;
+ }
+
+ private String getBaseUrl() {
+ Set<String> nodes = solrCloudClient.getClusterStateProvider().getLiveNodes();
+ String[] nodeArr = nodes.toArray(new String[0]);
+ // getting nodes URL as 'port_solr', so converting it to 'port/solr'
+ return nodeArr[0].replaceAll("_", "/");
+ }
+
}
diff --git a/security-admin/scripts/install.properties b/security-admin/scripts/install.properties
index ae7a00d..5a8b00c 100644
--- a/security-admin/scripts/install.properties
+++ b/security-admin/scripts/install.properties
@@ -106,6 +106,7 @@ audit_solr_zookeepers=
audit_solr_collection_name=ranger_audits
#solr Properties for cloud mode
audit_solr_config_name=ranger_audits
+audit_solr_configset_location=
audit_solr_no_shards=1
audit_solr_no_replica=1
audit_solr_max_shards_per_node=1
diff --git a/security-admin/scripts/setup.sh b/security-admin/scripts/setup.sh
index f6f6f56..c3f51a0 100755
--- a/security-admin/scripts/setup.sh
+++ b/security-admin/scripts/setup.sh
@@ -156,6 +156,7 @@ lookup_keytab=$(get_prop 'lookup_keytab' $PROPFILE)
hadoop_conf=$(get_prop 'hadoop_conf' $PROPFILE)
audit_solr_collection_name=$(get_prop 'audit_solr_collection_name' $PROPFILE)
audit_solr_config_name=$(get_prop 'audit_solr_config_name' $PROPFILE)
+audit_solr_configset_location=$(get_prop 'audit_solr_configset_location' $PROPFILE)
audit_solr_no_shards=$(get_prop 'audit_solr_no_shards' $PROPFILE)
audit_solr_no_replica=$(get_prop 'audit_solr_no_replica' $PROPFILE)
audit_solr_max_shards_per_node=$(get_prop 'audit_solr_max_shards_per_node' $PROPFILE)
@@ -476,6 +477,13 @@ update_properties() {
updatePropertyToFilePy $propertyName $newPropertyValue $to_file_ranger
fi
+ if [ "${audit_solr_configset_location}" != "" ]
+ then
+ propertyName=ranger.audit.solr.configset.location
+ newPropertyValue="${audit_solr_configset_location}"
+ updatePropertyToFilePy $propertyName $newPropertyValue $to_file_ranger
+ fi
+
if [ "${audit_solr_no_shards}" != "" ]
then
propertyName=ranger.audit.solr.no.shards
diff --git a/security-admin/src/main/resources/conf.dist/ranger-admin-site.xml b/security-admin/src/main/resources/conf.dist/ranger-admin-site.xml
index 793c479..d32a324 100644
--- a/security-admin/src/main/resources/conf.dist/ranger-admin-site.xml
+++ b/security-admin/src/main/resources/conf.dist/ranger-admin-site.xml
@@ -341,6 +341,10 @@
<value></value>
</property>
<property>
+ <name>ranger.audit.solr.configset.location</name>
+ <value></value>
+ </property>
+ <property>
<name>ranger.audit.solr.no.shards</name>
<value></value>
</property>