You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by GitBox <gi...@apache.org> on 2022/09/21 01:25:28 UTC
[GitHub] [bigtop] sekikn opened a new pull request, #1020: BIGTOP-3820. Upgrade Ranger to 2.3.0.
sekikn opened a new pull request, #1020:
URL: https://github.com/apache/bigtop/pull/1020
<!--
Thanks for sending a pull request!
1. If this is your first time, please read our contributor guidelines: https://cwiki.apache.org/confluence/display/BIGTOP/How+to+Contribute
2. Make sure your PR title starts with JIRA issue id, e.g., 'BIGTOP-3638: Your PR title ...'.
-->
### Description of PR
This PR upgrades Ranger to 2.3.0 and add the following improvements:
* Remove unnecessary files from RPM so that setup after installation works
* Rename the patch related to RANGER-3818
* Add "AutoReq: no" to the spec file for avoiding the "nothing provides /usr/bin/python" errors
* Add some options to the mvn options to accelerate the build
* Replace the dependency on sh-utils with coreutils, due to the same reason as BIGTOP-3377
* Fix typos in ranger.spec
### How was this patch tested?
I ensured that this PR works on CentOS 7 through the following steps. I'll turn them into puppet manifests and smoke tests in BIGTOP-3702 and BIGTOP-3703 respectively later.
1. Build the Ranger RPM:
```
$ ./gradlew ranger-clean ranger-pkg repo -Dbuildwithdeps=true
...
BUILD SUCCESSFUL in 16m 14s
34 actionable tasks: 34 executed
```
2. Install and setup Ranger Admin:
```
$ sudo -i
# yum install -y java-1.8.0-openjdk-devel postgresql-jdbc postgresql-server python3
# sudo -u postgres pg_ctl initdb -D /var/lib/pgsql/data
# systemctl start postgresql.service
# yum install -y yum-utils
# yum-config-manager --add-repo file://$PWD/output
# yum update -y
# yum install -y --nogpgcheck bigtop-utils ranger-admin
# cd /usr/lib/ranger-admin
# sed -i \
> -e 's|^\(DB_FLAVOR\)=\(.*\)|\1=POSTGRES|' \
> -e 's|^\(SQL_CONNECTOR_JAR\)=\(.*\)|\1=/usr/share/java/postgresql-jdbc.jar|' \
> -e 's|^\(db_root_user\)=\(.*\)|\1=postgres|' \
> -e 's|^\(db_root_password\)=\(.*\)|\1=|' \
> -e 's|^\(db_host\)=\(.*\)|\1=localhost|' \
> -e 's|^\(db_name\)=\(.*\)|\1=ranger|' \
> -e 's|^\(db_user\)=\(.*\)|\1=rangeradmin|' \
> -e 's|^\(db_password\)=\(.*\)|\1=Admin01234|' \
> -e 's|^\(rangerAdmin_password\)=\(.*\)|\1=Admin01234|' \
> -e 's|^\(rangerUsersync_password\)=\(.*\)|\1=Admin01234|' \
> -e 's|^\(rangerTagsync_password\)=\(.*\)|\1=Admin01234|' \
> -e 's|^\(keyadmin_password\)=\(.*\)|\1=Admin01234|' \
> -e 's|^\(audit_store\)=\(.*\)|\1=|' \
> install.properties
# . /usr/lib/bigtop-utils/bigtop-detect-javahome
# ./setup.sh
# ./set_globals.sh
# systemctl daemon-reload
# systemctl start ranger-admin
```
3. Add HDFS policy called "hadoopdev" via Ranger UI
4. Install Ranger HDFS plugin (assuming HDFS is already installed here):
```
# yum install -y --nogpgcheck ranger-hdfs-plugin
# cd /usr/lib/ranger-hdfs-plugin
# sed -i \
> -e 's|^\(POLICY_MGR_URL\) *= *\(.*\)|\1=http://localhost:6080|' \
> -e 's|^\(REPOSITORY_NAME\) *= *\(.*\)|\1=hadoopdev|' \
> -e 's|^\(COMPONENT_INSTALL_DIR_NAME\) *= *\(.*\)|\1=/usr/lib/hadoop|' \
> install.properties
# cp /usr/lib/hadoop/lib/hadoop-shaded-guava-1.1.1.jar install/lib
# . /usr/lib/bigtop-utils/bigtop-detect-javahome
# ./enable-hdfs-plugin.sh
# cp /usr/lib/ranger-admin/ews/webapp/WEB-INF/lib/jersey-client-1.19.jar /usr/lib/hadoop/lib
# systemctl restart hadoop-hdfs-namenode
# systemctl restart hadoop-hdfs-datanode
```
5. Confirm the "hadoopdev" policy works
### For code changes:
- [x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'BIGTOP-3638. Your PR title ...')?
- [x] Make sure that newly added files do not have any licensing issues. When in doubt refer to https://www.apache.org/licenses/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] sekikn merged pull request #1020: BIGTOP-3820. Upgrade Ranger to 2.3.0.
Posted by GitBox <gi...@apache.org>.
sekikn merged PR #1020:
URL: https://github.com/apache/bigtop/pull/1020
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] sekikn commented on pull request #1020: BIGTOP-3820. Upgrade Ranger to 2.3.0.
Posted by GitBox <gi...@apache.org>.
sekikn commented on PR #1020:
URL: https://github.com/apache/bigtop/pull/1020#issuecomment-1272423157
Merged into master. Thank you for the review @iwasakims!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org