You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Andrew Wong (Code Review)" <ge...@cloudera.org> on 2020/03/07 04:30:57 UTC
[kudu-CR] [wip] ranger: set up MiniRanger
Andrew Wong has uploaded this change for review. ( http://gerrit.cloudera.org:8080/15385
Change subject: [wip] ranger: set up MiniRanger
......................................................................
[wip] ranger: set up MiniRanger
Got to a point where I successfully create the Ranger database in
Postgres, but haven't tried making any calls in Ranger yet.
Also configs will likely need to be adjusted for some reason or other --
they're pretty minimal at the moment.
Also haven't found a good way to initialize things without having to
'sudo -u postgres' which doesn't feel great.
After running StartStopRanger, I can run the following locally:
[awong@va1022 debug]$ sudo -u postgres ./bin/postgres/psql -p <port>
ranger=# \c ranger
psql (12.2)
Type "help" for help.
ranger=# \c ranger
You are now connected to database "ranger" as user "postgres".
ranger=# \d
List of relations
Schema | Name | Type | Owner
--------+--------------------------------+----------+-------------
public | vx_trx_log | view | rangeradmin
public | x_access_type_def | table | rangeradmin
public | x_access_type_def_grants | table | rangeradmin
public | x_access_type_def_grants_seq | sequence | rangeradmin
public | x_access_type_def_seq | sequence | rangeradmin
public | x_asset | table | rangeradmin
public | x_asset_seq | sequence | rangeradmin
public | x_audit_map | table | rangeradmin
public | x_audit_map_seq | sequence | rangeradmin
public | x_auth_sess | table | rangeradmin
public | x_auth_sess_seq | sequence | rangeradmin
public | x_context_enricher_def | table | rangeradmin
public | x_context_enricher_def_seq | sequence | rangeradmin
public | x_cred_store | table | rangeradmin
public | x_cred_store_seq | sequence | rangeradmin
public | x_data_hist | table | rangeradmin
public | x_data_hist_seq | sequence | rangeradmin
public | x_datamask_type_def | table | rangeradmin
public | x_datamask_type_def_seq | sequence | rangeradmin
Change-Id: I95574b2a8359be3437c0993dd5ce63479fd71548
---
M src/kudu/ranger/CMakeLists.txt
M src/kudu/ranger/mini_ranger-test.cc
M src/kudu/ranger/mini_ranger.cc
M src/kudu/ranger/mini_ranger.h
M src/kudu/util/subprocess.cc
M thirdparty/build-thirdparty.sh
6 files changed, 162 insertions(+), 128 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/85/15385/1
--
To view, visit http://gerrit.cloudera.org:8080/15385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I95574b2a8359be3437c0993dd5ce63479fd71548
Gerrit-Change-Number: 15385
Gerrit-PatchSet: 1
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
[kudu-CR] [wip] ranger: set up MiniRanger
Posted by "Attila Bukor (Code Review)" <ge...@cloudera.org>.
Attila Bukor has uploaded a new patch set (#5) to the change originally created by Andrew Wong. ( http://gerrit.cloudera.org:8080/15385 )
Change subject: [wip] ranger: set up MiniRanger
......................................................................
[wip] ranger: set up MiniRanger
I'm opting to not use the setup.sh script [1] included with Ranger since
it touches various system-wide files in /etc. Instead, I'm drawing
inspiration from Impala's Ranger setup [2].
The actual starting of the Ranger admin service is inspired by the
script included with Ranger [3], which I opted to not use mostly for
aesthetic reasons.
Got to a point where I successfully create the Ranger database in
Postgres, but I haven't been able to use open the Ranger web UI locally,
so it looks these configs are insufficient.
After running TestStartStopRanger, I can run the following locally:
[awong@va1022 debug]$ sudo -u postgres ./bin/postgres/psql -p <port>
ranger=# \c ranger
psql (12.2)
Type "help" for help.
ranger=# \c ranger
You are now connected to database "ranger" as user "postgres".
ranger=# \d
List of relations
Schema | Name | Type | Owner
--------+--------------------------------+----------+-------------
public | vx_trx_log | view | rangeradmin
public | x_access_type_def | table | rangeradmin
public | x_access_type_def_grants | table | rangeradmin
public | x_access_type_def_grants_seq | sequence | rangeradmin
public | x_access_type_def_seq | sequence | rangeradmin
public | x_asset | table | rangeradmin
public | x_asset_seq | sequence | rangeradmin
public | x_audit_map | table | rangeradmin
public | x_audit_map_seq | sequence | rangeradmin
public | x_auth_sess | table | rangeradmin
public | x_auth_sess_seq | sequence | rangeradmin
public | x_context_enricher_def | table | rangeradmin
public | x_context_enricher_def_seq | sequence | rangeradmin
public | x_cred_store | table | rangeradmin
public | x_cred_store_seq | sequence | rangeradmin
public | x_data_hist | table | rangeradmin
public | x_data_hist_seq | sequence | rangeradmin
public | x_datamask_type_def | table | rangeradmin
public | x_datamask_type_def_seq | sequence | rangeradmin
[1] https://github.com/apache/ranger/blob/master/security-admin/scripts/setup.sh
[2] https://github.com/apache/impala/commit/0cb7187841780cabe368607ff559e493be59db22
[3] https://github.com/apache/ranger/blob/master/embeddedwebserver/scripts/ranger-admin-services.sh
Change-Id: I95574b2a8359be3437c0993dd5ce63479fd71548
---
M src/kudu/ranger/CMakeLists.txt
A src/kudu/ranger/mini_postgres.cc
A src/kudu/ranger/mini_postgres.h
M src/kudu/ranger/mini_ranger-test.cc
M src/kudu/ranger/mini_ranger.cc
M src/kudu/ranger/mini_ranger.h
M src/kudu/util/subprocess.cc
M thirdparty/build-thirdparty.sh
8 files changed, 1,253 insertions(+), 168 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/85/15385/5
--
To view, visit http://gerrit.cloudera.org:8080/15385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I95574b2a8359be3437c0993dd5ce63479fd71548
Gerrit-Change-Number: 15385
Gerrit-PatchSet: 5
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] [wip] ranger: set up MiniRanger
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/15385
to look at the new patch set (#3).
Change subject: [wip] ranger: set up MiniRanger
......................................................................
[wip] ranger: set up MiniRanger
I'm opting to not use the setup.sh script [1] included with Ranger since
it touches various system-wide files in /etc. Instead, I'm drawing
inspiration from Impala's Ranger setup [2].
The actual starting of the Ranger admin service is inspired by the
script included with Ranger [3], which I opted to not use mostly for
aesthetic reasons.
Got to a point where I successfully create the Ranger database in
Postgres, but haven't tried making any calls in Ranger yet.
Also configs will likely need to be adjusted for some reason or other --
they're pretty minimal at the moment.
Also haven't found a good way to initialize things without having to
'sudo -u postgres' which doesn't feel great.
After running StartStopRanger, I can run the following locally:
[awong@va1022 debug]$ sudo -u postgres ./bin/postgres/psql -p <port>
ranger=# \c ranger
psql (12.2)
Type "help" for help.
ranger=# \c ranger
You are now connected to database "ranger" as user "postgres".
ranger=# \d
List of relations
Schema | Name | Type | Owner
--------+--------------------------------+----------+-------------
public | vx_trx_log | view | rangeradmin
public | x_access_type_def | table | rangeradmin
public | x_access_type_def_grants | table | rangeradmin
public | x_access_type_def_grants_seq | sequence | rangeradmin
public | x_access_type_def_seq | sequence | rangeradmin
public | x_asset | table | rangeradmin
public | x_asset_seq | sequence | rangeradmin
public | x_audit_map | table | rangeradmin
public | x_audit_map_seq | sequence | rangeradmin
public | x_auth_sess | table | rangeradmin
public | x_auth_sess_seq | sequence | rangeradmin
public | x_context_enricher_def | table | rangeradmin
public | x_context_enricher_def_seq | sequence | rangeradmin
public | x_cred_store | table | rangeradmin
public | x_cred_store_seq | sequence | rangeradmin
public | x_data_hist | table | rangeradmin
public | x_data_hist_seq | sequence | rangeradmin
public | x_datamask_type_def | table | rangeradmin
public | x_datamask_type_def_seq | sequence | rangeradmin
[1] https://github.com/apache/ranger/blob/master/security-admin/scripts/setup.sh
[2] https://github.com/apache/impala/commit/0cb7187841780cabe368607ff559e493be59db22
[3] https://github.com/apache/ranger/blob/master/embeddedwebserver/scripts/ranger-admin-services.sh
Change-Id: I95574b2a8359be3437c0993dd5ce63479fd71548
---
M src/kudu/ranger/CMakeLists.txt
A src/kudu/ranger/mini_postgres.cc
A src/kudu/ranger/mini_postgres.h
M src/kudu/ranger/mini_ranger-test.cc
M src/kudu/ranger/mini_ranger.cc
M src/kudu/ranger/mini_ranger.h
M src/kudu/util/subprocess.cc
M thirdparty/build-thirdparty.sh
8 files changed, 384 insertions(+), 128 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/85/15385/3
--
To view, visit http://gerrit.cloudera.org:8080/15385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I95574b2a8359be3437c0993dd5ce63479fd71548
Gerrit-Change-Number: 15385
Gerrit-PatchSet: 3
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] [wip] ranger: set up MiniRanger
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Alexey Serbin, Kudu Jenkins, Hao Hao,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/15385
to look at the new patch set (#4).
Change subject: [wip] ranger: set up MiniRanger
......................................................................
[wip] ranger: set up MiniRanger
I'm opting to not use the setup.sh script [1] included with Ranger since
it touches various system-wide files in /etc. Instead, I'm drawing
inspiration from Impala's Ranger setup [2].
The actual starting of the Ranger admin service is inspired by the
script included with Ranger [3], which I opted to not use mostly for
aesthetic reasons.
Got to a point where I successfully create the Ranger database in
Postgres, but I haven't been able to use open the Ranger web UI locally,
so it looks these configs are insufficient.
After running TestStartStopRanger, I can run the following locally:
[awong@va1022 debug]$ sudo -u postgres ./bin/postgres/psql -p <port>
ranger=# \c ranger
psql (12.2)
Type "help" for help.
ranger=# \c ranger
You are now connected to database "ranger" as user "postgres".
ranger=# \d
List of relations
Schema | Name | Type | Owner
--------+--------------------------------+----------+-------------
public | vx_trx_log | view | rangeradmin
public | x_access_type_def | table | rangeradmin
public | x_access_type_def_grants | table | rangeradmin
public | x_access_type_def_grants_seq | sequence | rangeradmin
public | x_access_type_def_seq | sequence | rangeradmin
public | x_asset | table | rangeradmin
public | x_asset_seq | sequence | rangeradmin
public | x_audit_map | table | rangeradmin
public | x_audit_map_seq | sequence | rangeradmin
public | x_auth_sess | table | rangeradmin
public | x_auth_sess_seq | sequence | rangeradmin
public | x_context_enricher_def | table | rangeradmin
public | x_context_enricher_def_seq | sequence | rangeradmin
public | x_cred_store | table | rangeradmin
public | x_cred_store_seq | sequence | rangeradmin
public | x_data_hist | table | rangeradmin
public | x_data_hist_seq | sequence | rangeradmin
public | x_datamask_type_def | table | rangeradmin
public | x_datamask_type_def_seq | sequence | rangeradmin
[1] https://github.com/apache/ranger/blob/master/security-admin/scripts/setup.sh
[2] https://github.com/apache/impala/commit/0cb7187841780cabe368607ff559e493be59db22
[3] https://github.com/apache/ranger/blob/master/embeddedwebserver/scripts/ranger-admin-services.sh
Change-Id: I95574b2a8359be3437c0993dd5ce63479fd71548
---
M src/kudu/ranger/CMakeLists.txt
A src/kudu/ranger/mini_postgres.cc
A src/kudu/ranger/mini_postgres.h
M src/kudu/ranger/mini_ranger-test.cc
M src/kudu/ranger/mini_ranger.cc
M src/kudu/ranger/mini_ranger.h
M src/kudu/util/subprocess.cc
M thirdparty/build-thirdparty.sh
8 files changed, 1,255 insertions(+), 168 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/85/15385/4
--
To view, visit http://gerrit.cloudera.org:8080/15385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I95574b2a8359be3437c0993dd5ce63479fd71548
Gerrit-Change-Number: 15385
Gerrit-PatchSet: 4
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] [wip] ranger: set up MiniRanger
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/15385
to look at the new patch set (#2).
Change subject: [wip] ranger: set up MiniRanger
......................................................................
[wip] ranger: set up MiniRanger
Got to a point where I successfully create the Ranger database in
Postgres, but haven't tried making any calls in Ranger yet.
Also configs will likely need to be adjusted for some reason or other --
they're pretty minimal at the moment.
Also haven't found a good way to initialize things without having to
'sudo -u postgres' which doesn't feel great.
After running StartStopRanger, I can run the following locally:
[awong@va1022 debug]$ sudo -u postgres ./bin/postgres/psql -p <port>
ranger=# \c ranger
psql (12.2)
Type "help" for help.
ranger=# \c ranger
You are now connected to database "ranger" as user "postgres".
ranger=# \d
List of relations
Schema | Name | Type | Owner
--------+--------------------------------+----------+-------------
public | vx_trx_log | view | rangeradmin
public | x_access_type_def | table | rangeradmin
public | x_access_type_def_grants | table | rangeradmin
public | x_access_type_def_grants_seq | sequence | rangeradmin
public | x_access_type_def_seq | sequence | rangeradmin
public | x_asset | table | rangeradmin
public | x_asset_seq | sequence | rangeradmin
public | x_audit_map | table | rangeradmin
public | x_audit_map_seq | sequence | rangeradmin
public | x_auth_sess | table | rangeradmin
public | x_auth_sess_seq | sequence | rangeradmin
public | x_context_enricher_def | table | rangeradmin
public | x_context_enricher_def_seq | sequence | rangeradmin
public | x_cred_store | table | rangeradmin
public | x_cred_store_seq | sequence | rangeradmin
public | x_data_hist | table | rangeradmin
public | x_data_hist_seq | sequence | rangeradmin
public | x_datamask_type_def | table | rangeradmin
public | x_datamask_type_def_seq | sequence | rangeradmin
Change-Id: I95574b2a8359be3437c0993dd5ce63479fd71548
---
M src/kudu/ranger/CMakeLists.txt
A src/kudu/ranger/mini_postgres.cc
A src/kudu/ranger/mini_postgres.h
M src/kudu/ranger/mini_ranger-test.cc
M src/kudu/ranger/mini_ranger.cc
M src/kudu/ranger/mini_ranger.h
M src/kudu/util/subprocess.cc
M thirdparty/build-thirdparty.sh
8 files changed, 384 insertions(+), 128 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/85/15385/2
--
To view, visit http://gerrit.cloudera.org:8080/15385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I95574b2a8359be3437c0993dd5ce63479fd71548
Gerrit-Change-Number: 15385
Gerrit-PatchSet: 2
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] [wip] ranger: set up MiniRanger
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/15385 )
Change subject: [wip] ranger: set up MiniRanger
......................................................................
Patch Set 3:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/15385/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/15385/3//COMMIT_MSG@23
PS3, Line 23: Also haven't found a good way to initialize things without having to
: 'sudo -u postgres' which doesn't feel great.
Could this help:
https://www.postgresql.org/message-id/31725673.36.1288780165320.JavaMail.root%40www
?
PostgreSQL creates a role/user inside the cluster with the same name as the user that ran initdb.
http://gerrit.cloudera.org:8080/#/c/15385/3/src/kudu/ranger/mini_postgres.h
File src/kudu/ranger/mini_postgres.h:
http://gerrit.cloudera.org:8080/#/c/15385/3/src/kudu/ranger/mini_postgres.h@62
PS3, Line 62: pg_process_
nit: since now it's just a wrapper around postgresql, rename into 'process_'?
http://gerrit.cloudera.org:8080/#/c/15385/3/src/kudu/ranger/mini_postgres.cc
File src/kudu/ranger/mini_postgres.cc:
http://gerrit.cloudera.org:8080/#/c/15385/3/src/kudu/ranger/mini_postgres.cc@111
PS3, Line 111: "sudo", "-u", "postgres",
Is it possible to avoid using a special user for that? What prevents to run postgres under the OS user that runs all the commands here?
--
To view, visit http://gerrit.cloudera.org:8080/15385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I95574b2a8359be3437c0993dd5ce63479fd71548
Gerrit-Change-Number: 15385
Gerrit-PatchSet: 3
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Sat, 07 Mar 2020 07:57:04 +0000
Gerrit-HasComments: Yes
[kudu-CR] [wip] ranger: set up MiniRanger
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has abandoned this change. ( http://gerrit.cloudera.org:8080/15385 )
Change subject: [wip] ranger: set up MiniRanger
......................................................................
Abandoned
Superceded by Attila's patches.
--
To view, visit http://gerrit.cloudera.org:8080/15385
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: abandon
Gerrit-Change-Id: I95574b2a8359be3437c0993dd5ce63479fd71548
Gerrit-Change-Number: 15385
Gerrit-PatchSet: 5
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)