You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Austin Nobis (Code Review)" <ge...@cloudera.org> on 2019/04/02 23:19:07 UTC

[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger 

Austin Nobis has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12914


Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger
......................................................................

IMPALA-8226: Add grant/revoke to/from group for Ranger

This patch adds fupport for GRANT privilege statements to GROUP and
REVOKE privilege statements from GROUP.  The grammar has been updated to
support FROM GROUP and TO GROUP for GRANT/REVOKE statements, i.e:

GRANT <privilege> ON <resource> TO GROUP <group>
REVOKE <privilege> ON <resource> FROM GROUP <group>

Currently, only Ranger's authorization implementation supports GROUP
based privileges. Sentry will throw an UnsupportedOperationException if
it is the enabled authorization provider and this new grammar is used.

Testing:
- AuthorizationStmtTest was updated to also test for GROUP
  authorization.
- ToSqlTest was updated to test for GROUP changes to the grammar.
- A GROUP based E2E test was added to test_ranger.py
- Ran all FE tests
- Ran authorization E2E tests

Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
---
M common/thrift/CatalogObjects.thrift
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/org/apache/impala/authorization/AuthorizationManager.java
M fe/src/main/java/org/apache/impala/authorization/NoneAuthorizationFactory.java
M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java
M fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java
M fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java
M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java
M fe/src/test/resources/ranger-hive-security.xml
M testdata/bin/create-load-data.sh
A testdata/cluster/ranger/setup/impala_group.json.template
M testdata/cluster/ranger/setup/impala_user.json.template
M tests/authorization/test_ranger.py
15 files changed, 212 insertions(+), 83 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/12914/2
-- 
To view, visit http://gerrit.cloudera.org:8080/12914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
Gerrit-Change-Number: 12914
Gerrit-PatchSet: 2
Gerrit-Owner: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>

[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger

Posted by "Austin Nobis (Code Review)" <ge...@cloudera.org>.
Austin Nobis has uploaded a new patch set (#8). ( http://gerrit.cloudera.org:8080/12914 )

Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger
......................................................................

IMPALA-8226: Add grant/revoke to/from group for Ranger

This patch adds fupport for GRANT privilege statements to GROUP and
REVOKE privilege statements from GROUP.  The grammar has been updated to
support FROM GROUP and TO GROUP for GRANT/REVOKE statements, i.e:

GRANT <privilege> ON <resource> TO GROUP <group>
REVOKE <privilege> ON <resource> FROM GROUP <group>

Currently, only Ranger's authorization implementation supports GROUP
based privileges. Sentry will throw an UnsupportedOperationException if
it is the enabled authorization provider and this new grammar is used.

Testing:
- AuthorizationStmtTest was updated to also test for GROUP
  authorization.
- ToSqlTest was updated to test for GROUP changes to the grammar.
- A GROUP based E2E test was added to test_ranger.py
- ParserTest was updated to test combinations for GrantRevokePrivilege
- AnalyzeAuthStmtsTest was updated to test for USER and GROUP identities
- Ran all FE tests
- Ran authorization E2E tests

Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
---
M common/thrift/CatalogObjects.thrift
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/org/apache/impala/authorization/AuthorizationManager.java
M fe/src/main/java/org/apache/impala/authorization/NoneAuthorizationFactory.java
M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java
M fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java
M fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java
M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java
M fe/src/test/java/org/apache/impala/analysis/ParserTest.java
M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java
M fe/src/test/resources/ranger-hive-security.xml
M testdata/bin/create-load-data.sh
A testdata/cluster/ranger/setup/impala_group.json.template
M testdata/cluster/ranger/setup/impala_user.json.template
M tests/authorization/test_ranger.py
17 files changed, 470 insertions(+), 323 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/12914/8
-- 
To view, visit http://gerrit.cloudera.org:8080/12914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
Gerrit-Change-Number: 12914
Gerrit-PatchSet: 8
Gerrit-Owner: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>

[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger

Posted by "Austin Nobis (Code Review)" <ge...@cloudera.org>.
Austin Nobis has posted comments on this change. ( http://gerrit.cloudera.org:8080/12914 )

Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger
......................................................................


Patch Set 9:

Fixed an issue caused by the group configuration on the Jenkins host that caused the merge to fail.

Tested here: https://master-02.jenkins.cloudera.com/view/Impala/view/Private/job/impala-private-parameterized/4690/


-- 
To view, visit http://gerrit.cloudera.org:8080/12914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
Gerrit-Change-Number: 12914
Gerrit-PatchSet: 9
Gerrit-Owner: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Thu, 04 Apr 2019 18:56:42 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12914 )

Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger
......................................................................


Patch Set 11: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/12914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
Gerrit-Change-Number: 12914
Gerrit-PatchSet: 11
Gerrit-Owner: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Fri, 05 Apr 2019 00:04:35 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger

Posted by "Austin Nobis (Code Review)" <ge...@cloudera.org>.
Austin Nobis has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/12914 )

Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger
......................................................................

IMPALA-8226: Add grant/revoke to/from group for Ranger

This patch adds fupport for GRANT privilege statements to GROUP and
REVOKE privilege statements from GROUP.  The grammar has been updated to
support FROM GROUP and TO GROUP for GRANT/REVOKE statements, i.e:

GRANT <privilege> ON <resource> TO GROUP <group>
REVOKE <privilege> ON <resource> FROM GROUP <group>

Currently, only Ranger's authorization implementation supports GROUP
based privileges. Sentry will throw an UnsupportedOperationException if
it is the enabled authorization provider and this new grammar is used.

Testing:
- AuthorizationStmtTest was updated to also test for GROUP
  authorization.
- ToSqlTest was updated to test for GROUP changes to the grammar.
- A GROUP based E2E test was added to test_ranger.py
- ParserTest was updated to test combinations for GrantRevokePrivilege
- AnalyzeAuthStmtsTest was updated to test for USER and GROUP identities
- Ran all FE tests
- Ran authorization E2E tests

Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
---
M common/thrift/CatalogObjects.thrift
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/org/apache/impala/authorization/AuthorizationManager.java
M fe/src/main/java/org/apache/impala/authorization/NoneAuthorizationFactory.java
M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java
M fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java
M fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java
M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java
M fe/src/test/java/org/apache/impala/analysis/ParserTest.java
M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java
M fe/src/test/resources/ranger-hive-security.xml
M testdata/bin/create-load-data.sh
A testdata/cluster/ranger/setup/impala_group.json.template
M testdata/cluster/ranger/setup/impala_user.json.template
M tests/authorization/test_ranger.py
17 files changed, 467 insertions(+), 321 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/12914/5
-- 
To view, visit http://gerrit.cloudera.org:8080/12914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
Gerrit-Change-Number: 12914
Gerrit-PatchSet: 5
Gerrit-Owner: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>

[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger

Posted by "Fredy Wijaya (Code Review)" <ge...@cloudera.org>.
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/12914 )

Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger
......................................................................


Patch Set 3:

(2 comments)

Can you add tests in ParserTest and AnalyzeAuthStmtTest?

http://gerrit.cloudera.org:8080/#/c/12914/3/fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java
File fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java:

http://gerrit.cloudera.org:8080/#/c/12914/3/fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java@3002
PS3, Line 3002: private class WithRanger implements WithPrincipal
Instead of having a boolean flag, let's use WithRangerGroup and rename this class with WithRangerUser. It think it's much cleaner.


http://gerrit.cloudera.org:8080/#/c/12914/3/tests/authorization/test_ranger.py
File tests/authorization/test_ranger.py:

http://gerrit.cloudera.org:8080/#/c/12914/3/tests/authorization/test_ranger.py@68
PS3, Line 68: time.sleep(35)
since you changed the polling interval to 5 seconds, we no longer need to wait that long.



-- 
To view, visit http://gerrit.cloudera.org:8080/12914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
Gerrit-Change-Number: 12914
Gerrit-PatchSet: 3
Gerrit-Owner: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Wed, 03 Apr 2019 15:58:22 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12914 )

Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger
......................................................................

IMPALA-8226: Add grant/revoke to/from group for Ranger

This patch adds fupport for GRANT privilege statements to GROUP and
REVOKE privilege statements from GROUP.  The grammar has been updated to
support FROM GROUP and TO GROUP for GRANT/REVOKE statements, i.e:

GRANT <privilege> ON <resource> TO GROUP <group>
REVOKE <privilege> ON <resource> FROM GROUP <group>

Currently, only Ranger's authorization implementation supports GROUP
based privileges. Sentry will throw an UnsupportedOperationException if
it is the enabled authorization provider and this new grammar is used.

Testing:
- AuthorizationStmtTest was updated to also test for GROUP
  authorization.
- ToSqlTest was updated to test for GROUP changes to the grammar.
- A GROUP based E2E test was added to test_ranger.py
- ParserTest was updated to test combinations for GrantRevokePrivilege
- AnalyzeAuthStmtsTest was updated to test for USER and GROUP identities
- Ran all FE tests
- Ran authorization E2E tests

Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
Reviewed-on: http://gerrit.cloudera.org:8080/12914
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M common/thrift/CatalogObjects.thrift
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/org/apache/impala/authorization/AuthorizationManager.java
M fe/src/main/java/org/apache/impala/authorization/NoneAuthorizationFactory.java
M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java
M fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java
M fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java
M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java
M fe/src/test/java/org/apache/impala/analysis/ParserTest.java
M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java
M fe/src/test/resources/ranger-hive-security.xml
M testdata/bin/create-load-data.sh
A testdata/cluster/ranger/setup/impala_group.json.template
M testdata/cluster/ranger/setup/impala_user.json.template
M tests/authorization/test_ranger.py
17 files changed, 461 insertions(+), 323 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/12914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
Gerrit-Change-Number: 12914
Gerrit-PatchSet: 12
Gerrit-Owner: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>

[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12914 )

Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger
......................................................................


Patch Set 7: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/3977/


-- 
To view, visit http://gerrit.cloudera.org:8080/12914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
Gerrit-Change-Number: 12914
Gerrit-PatchSet: 7
Gerrit-Owner: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Thu, 04 Apr 2019 01:55:57 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12914 )

Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger
......................................................................


Patch Set 9:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/2647/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/12914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
Gerrit-Change-Number: 12914
Gerrit-PatchSet: 9
Gerrit-Owner: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Thu, 04 Apr 2019 19:21:41 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12914 )

Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger
......................................................................


Patch Set 2:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/12914/2/testdata/bin/create-load-data.sh
File testdata/bin/create-load-data.sh:

http://gerrit.cloudera.org:8080/#/c/12914/2/testdata/bin/create-load-data.sh@305
PS2, Line 305:   
line has trailing whitespace


http://gerrit.cloudera.org:8080/#/c/12914/2/testdata/bin/create-load-data.sh@312
PS2, Line 312:   
line has trailing whitespace


http://gerrit.cloudera.org:8080/#/c/12914/2/tests/authorization/test_ranger.py
File tests/authorization/test_ranger.py:

http://gerrit.cloudera.org:8080/#/c/12914/2/tests/authorization/test_ranger.py@58
PS2, Line 58: ,
flake8: E501 line too long (91 > 90 characters)


http://gerrit.cloudera.org:8080/#/c/12914/2/tests/authorization/test_ranger.py@82
PS2, Line 82: ,
flake8: E501 line too long (91 > 90 characters)



-- 
To view, visit http://gerrit.cloudera.org:8080/12914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
Gerrit-Change-Number: 12914
Gerrit-PatchSet: 2
Gerrit-Owner: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Tue, 02 Apr 2019 23:20:09 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger

Posted by "Fredy Wijaya (Code Review)" <ge...@cloudera.org>.
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/12914 )

Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger
......................................................................


Patch Set 10: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/12914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
Gerrit-Change-Number: 12914
Gerrit-PatchSet: 10
Gerrit-Owner: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Thu, 04 Apr 2019 19:09:43 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12914 )

Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger
......................................................................


Patch Set 11: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/12914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
Gerrit-Change-Number: 12914
Gerrit-PatchSet: 11
Gerrit-Owner: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Thu, 04 Apr 2019 19:09:58 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger

Posted by "Austin Nobis (Code Review)" <ge...@cloudera.org>.
Austin Nobis has uploaded a new patch set (#6). ( http://gerrit.cloudera.org:8080/12914 )

Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger
......................................................................

IMPALA-8226: Add grant/revoke to/from group for Ranger

This patch adds fupport for GRANT privilege statements to GROUP and
REVOKE privilege statements from GROUP.  The grammar has been updated to
support FROM GROUP and TO GROUP for GRANT/REVOKE statements, i.e:

GRANT <privilege> ON <resource> TO GROUP <group>
REVOKE <privilege> ON <resource> FROM GROUP <group>

Currently, only Ranger's authorization implementation supports GROUP
based privileges. Sentry will throw an UnsupportedOperationException if
it is the enabled authorization provider and this new grammar is used.

Testing:
- AuthorizationStmtTest was updated to also test for GROUP
  authorization.
- ToSqlTest was updated to test for GROUP changes to the grammar.
- A GROUP based E2E test was added to test_ranger.py
- ParserTest was updated to test combinations for GrantRevokePrivilege
- AnalyzeAuthStmtsTest was updated to test for USER and GROUP identities
- Ran all FE tests
- Ran authorization E2E tests

Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
---
M common/thrift/CatalogObjects.thrift
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/org/apache/impala/authorization/AuthorizationManager.java
M fe/src/main/java/org/apache/impala/authorization/NoneAuthorizationFactory.java
M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java
M fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java
M fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java
M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java
M fe/src/test/java/org/apache/impala/analysis/ParserTest.java
M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java
M fe/src/test/resources/ranger-hive-security.xml
M testdata/bin/create-load-data.sh
A testdata/cluster/ranger/setup/impala_group.json.template
M testdata/cluster/ranger/setup/impala_user.json.template
M tests/authorization/test_ranger.py
17 files changed, 468 insertions(+), 323 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/12914/6
-- 
To view, visit http://gerrit.cloudera.org:8080/12914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
Gerrit-Change-Number: 12914
Gerrit-PatchSet: 6
Gerrit-Owner: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>

[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger

Posted by "Fredy Wijaya (Code Review)" <ge...@cloudera.org>.
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/12914 )

Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger
......................................................................


Patch Set 4:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/12914/4/fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java
File fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java:

http://gerrit.cloudera.org:8080/#/c/12914/4/fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java@165
PS4, Line 165:     String[] idents = {"myRole", "ROLE myRole", "GROUP myGroup", "USER myUser"};
             :     boolean[] isGrantVals = {true, false};
do we have tests for bad idents?


http://gerrit.cloudera.org:8080/#/c/12914/4/fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java
File fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java:

http://gerrit.cloudera.org:8080/#/c/12914/4/fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java@3153
PS4, Line 3153: withPrincipals.add((isUser_) ? new WithRangerUser() : new WithRangerGroup());
shouldn't we be running both user and ranger?


http://gerrit.cloudera.org:8080/#/c/12914/4/fe/src/test/java/org/apache/impala/analysis/ParserTest.java
File fe/src/test/java/org/apache/impala/analysis/ParserTest.java:

http://gerrit.cloudera.org:8080/#/c/12914/4/fe/src/test/java/org/apache/impala/analysis/ParserTest.java@3592
PS4, Line 3592:     String[] resources = {"SERVER", "SERVER foo", "DATABASE foo", "TABLE foo",
              :         "URI 'foo'"};
              :     String[] badResources = {"DATABASE", "TABLE", "URI", "URI foo", "TABLE 'foo'",
              :         "SERVER 'foo'", "DATABASE 'foo'"};
              :     String[] privileges = {"SELECT", "INSERT", "ALL", "REFRESH", "CREATE", "ALTER",
              :         "DROP"};
              :     String[] badPrivileges = {"UPDATE", "DELETE", "UPSERT", "FAKE"};
              :     String[] columnPrivResource = {"SELECT (a, b) ON TABLE foo", "SELECT () on TABLE foo",
              :         "INSERT (a, b) ON TABLE foo", "ALL (a, b) ON TABLE foo"};
              :     String[] badColumnPrivResource = {"SELECT (a,) ON TABLE foo",
              :         "SELECT (*) ON TABLE foo", "SELECT (a), b ON TABLE foo",
              :         "SELECT ((a)) ON TABLE foo", "SELECT (a, b) ON URI foo",
              :         "SELECT ON TABLE (a, b) foo",};
              :     String[][] grantRevoke = {{"GRANT", "TO"}, {"REVOKE", "FROM"}};
              :     String[] idents = {"myRole", "GROUP myGroup", "USER user", "ROLE myRole"};
              :     String[] badIdents = {"GROUP", "ROLE", "GROUP group", "GROUP role", "USER role",
              :         "FOOBAR foobar", ""};
this is a bit hard to read, maybe put each element in a new line where it makes sense?



-- 
To view, visit http://gerrit.cloudera.org:8080/12914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
Gerrit-Change-Number: 12914
Gerrit-PatchSet: 4
Gerrit-Owner: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Wed, 03 Apr 2019 19:06:36 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger

Posted by "Fredy Wijaya (Code Review)" <ge...@cloudera.org>.
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/12914 )

Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger
......................................................................


Patch Set 5:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/12914/5/common/thrift/CatalogObjects.thrift
File common/thrift/CatalogObjects.thrift:

http://gerrit.cloudera.org:8080/#/c/12914/5/common/thrift/CatalogObjects.thrift@485
PS5, Line 485: // Represents a principal type that maps to Sentry principal type.
             : // https://github.com/apache/sentry/blob/3d062f39ce6a047138660a7b3d0024bde916c5b4/sentry-service/sentry-service-api/src/gen/thrift/gen-javabean/org/apache/sentry/api/service/thrift/TSentryPrincipalType.java
nit: remove this comment i don't think this is specific to Sentry anymore.



-- 
To view, visit http://gerrit.cloudera.org:8080/12914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
Gerrit-Change-Number: 12914
Gerrit-PatchSet: 5
Gerrit-Owner: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Wed, 03 Apr 2019 20:54:24 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12914 )

Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger
......................................................................


Patch Set 3:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/2619/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/12914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
Gerrit-Change-Number: 12914
Gerrit-PatchSet: 3
Gerrit-Owner: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Wed, 03 Apr 2019 00:01:53 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12914 )

Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger
......................................................................


Patch Set 6:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/2627/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/12914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
Gerrit-Change-Number: 12914
Gerrit-PatchSet: 6
Gerrit-Owner: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Wed, 03 Apr 2019 21:50:26 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger

Posted by "Austin Nobis (Code Review)" <ge...@cloudera.org>.
Austin Nobis has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/12914 )

Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger
......................................................................

IMPALA-8226: Add grant/revoke to/from group for Ranger

This patch adds fupport for GRANT privilege statements to GROUP and
REVOKE privilege statements from GROUP.  The grammar has been updated to
support FROM GROUP and TO GROUP for GRANT/REVOKE statements, i.e:

GRANT <privilege> ON <resource> TO GROUP <group>
REVOKE <privilege> ON <resource> FROM GROUP <group>

Currently, only Ranger's authorization implementation supports GROUP
based privileges. Sentry will throw an UnsupportedOperationException if
it is the enabled authorization provider and this new grammar is used.

Testing:
- AuthorizationStmtTest was updated to also test for GROUP
  authorization.
- ToSqlTest was updated to test for GROUP changes to the grammar.
- A GROUP based E2E test was added to test_ranger.py
- ParserTest was updated to test combinations for GrantRevokePrivilege
- AnalyzeAuthStmtsTest was updated to test for USER and GROUP identities
- Ran all FE tests
- Ran authorization E2E tests

Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
---
M common/thrift/CatalogObjects.thrift
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/org/apache/impala/authorization/AuthorizationManager.java
M fe/src/main/java/org/apache/impala/authorization/NoneAuthorizationFactory.java
M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java
M fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java
M fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java
M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java
M fe/src/test/java/org/apache/impala/analysis/ParserTest.java
M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java
M fe/src/test/resources/ranger-hive-security.xml
M testdata/bin/create-load-data.sh
A testdata/cluster/ranger/setup/impala_group.json.template
M testdata/cluster/ranger/setup/impala_user.json.template
M tests/authorization/test_ranger.py
17 files changed, 470 insertions(+), 325 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/12914/4
-- 
To view, visit http://gerrit.cloudera.org:8080/12914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
Gerrit-Change-Number: 12914
Gerrit-PatchSet: 4
Gerrit-Owner: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>

[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger

Posted by "radford nguyen (Code Review)" <ge...@cloudera.org>.
radford nguyen has posted comments on this change. ( http://gerrit.cloudera.org:8080/12914 )

Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger
......................................................................


Patch Set 11:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/12914/7/fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java
File fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java:

http://gerrit.cloudera.org:8080/#/c/12914/7/fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java@193
PS7, Line 193:       String user, List<String> groups, String clusterName, List<TPrivilege> privileges) {
nit: We could probably use a `Collection<String> groups` to be more general here, since the group's items are copied into a `List` when creating the request.  Same with `privileges`.


http://gerrit.cloudera.org:8080/#/c/12914/7/fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java@233
PS7, Line 233:     if (!groups.isEmpty()) request.getGroups().addAll(groups);
nit: is the `if` statement really necessary given the contract of `addAll`?


http://gerrit.cloudera.org:8080/#/c/12914/10/fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java
File fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java:

http://gerrit.cloudera.org:8080/#/c/12914/10/fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java@170
PS10, Line 170:         "%s is not supported in Impalad", ClassUtil.getMethodName()));
Isn't it more accurate to say that this isn't supported with sentry?



-- 
To view, visit http://gerrit.cloudera.org:8080/12914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
Gerrit-Change-Number: 12914
Gerrit-PatchSet: 11
Gerrit-Owner: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Thu, 04 Apr 2019 19:57:09 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12914 )

Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger
......................................................................


Patch Set 10:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/2648/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/12914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
Gerrit-Change-Number: 12914
Gerrit-PatchSet: 10
Gerrit-Owner: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Thu, 04 Apr 2019 19:36:50 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12914 )

Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger
......................................................................


Patch Set 11:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3984/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/12914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
Gerrit-Change-Number: 12914
Gerrit-PatchSet: 11
Gerrit-Owner: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Thu, 04 Apr 2019 19:09:59 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger

Posted by "Austin Nobis (Code Review)" <ge...@cloudera.org>.
Austin Nobis has uploaded a new patch set (#9). ( http://gerrit.cloudera.org:8080/12914 )

Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger
......................................................................

IMPALA-8226: Add grant/revoke to/from group for Ranger

This patch adds fupport for GRANT privilege statements to GROUP and
REVOKE privilege statements from GROUP.  The grammar has been updated to
support FROM GROUP and TO GROUP for GRANT/REVOKE statements, i.e:

GRANT <privilege> ON <resource> TO GROUP <group>
REVOKE <privilege> ON <resource> FROM GROUP <group>

Currently, only Ranger's authorization implementation supports GROUP
based privileges. Sentry will throw an UnsupportedOperationException if
it is the enabled authorization provider and this new grammar is used.

Testing:
- AuthorizationStmtTest was updated to also test for GROUP
  authorization.
- ToSqlTest was updated to test for GROUP changes to the grammar.
- A GROUP based E2E test was added to test_ranger.py
- ParserTest was updated to test combinations for GrantRevokePrivilege
- AnalyzeAuthStmtsTest was updated to test for USER and GROUP identities
- Ran all FE tests
- Ran authorization E2E tests

Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
---
M common/thrift/CatalogObjects.thrift
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/org/apache/impala/authorization/AuthorizationManager.java
M fe/src/main/java/org/apache/impala/authorization/NoneAuthorizationFactory.java
M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java
M fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java
M fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java
M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java
M fe/src/test/java/org/apache/impala/analysis/ParserTest.java
M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java
M fe/src/test/resources/ranger-hive-security.xml
M testdata/bin/create-load-data.sh
A testdata/cluster/ranger/setup/impala_group.json.template
M testdata/cluster/ranger/setup/impala_user.json.template
M tests/authorization/test_ranger.py
17 files changed, 463 insertions(+), 323 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/12914/9
-- 
To view, visit http://gerrit.cloudera.org:8080/12914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
Gerrit-Change-Number: 12914
Gerrit-PatchSet: 9
Gerrit-Owner: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>

[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger

Posted by "Austin Nobis (Code Review)" <ge...@cloudera.org>.
Austin Nobis has uploaded a new patch set (#10). ( http://gerrit.cloudera.org:8080/12914 )

Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger
......................................................................

IMPALA-8226: Add grant/revoke to/from group for Ranger

This patch adds fupport for GRANT privilege statements to GROUP and
REVOKE privilege statements from GROUP.  The grammar has been updated to
support FROM GROUP and TO GROUP for GRANT/REVOKE statements, i.e:

GRANT <privilege> ON <resource> TO GROUP <group>
REVOKE <privilege> ON <resource> FROM GROUP <group>

Currently, only Ranger's authorization implementation supports GROUP
based privileges. Sentry will throw an UnsupportedOperationException if
it is the enabled authorization provider and this new grammar is used.

Testing:
- AuthorizationStmtTest was updated to also test for GROUP
  authorization.
- ToSqlTest was updated to test for GROUP changes to the grammar.
- A GROUP based E2E test was added to test_ranger.py
- ParserTest was updated to test combinations for GrantRevokePrivilege
- AnalyzeAuthStmtsTest was updated to test for USER and GROUP identities
- Ran all FE tests
- Ran authorization E2E tests

Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
---
M common/thrift/CatalogObjects.thrift
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/org/apache/impala/authorization/AuthorizationManager.java
M fe/src/main/java/org/apache/impala/authorization/NoneAuthorizationFactory.java
M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java
M fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java
M fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/test/java/org/apache/impala/analysis/AnalyzeAuthStmtsTest.java
M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java
M fe/src/test/java/org/apache/impala/analysis/ParserTest.java
M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java
M fe/src/test/resources/ranger-hive-security.xml
M testdata/bin/create-load-data.sh
A testdata/cluster/ranger/setup/impala_group.json.template
M testdata/cluster/ranger/setup/impala_user.json.template
M tests/authorization/test_ranger.py
17 files changed, 461 insertions(+), 323 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/12914/10
-- 
To view, visit http://gerrit.cloudera.org:8080/12914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
Gerrit-Change-Number: 12914
Gerrit-PatchSet: 10
Gerrit-Owner: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>

[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12914 )

Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger
......................................................................


Patch Set 4:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/2624/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/12914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
Gerrit-Change-Number: 12914
Gerrit-PatchSet: 4
Gerrit-Owner: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Wed, 03 Apr 2019 19:34:24 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12914 )

Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger
......................................................................


Patch Set 8:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/2639/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/12914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
Gerrit-Change-Number: 12914
Gerrit-PatchSet: 8
Gerrit-Owner: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Thu, 04 Apr 2019 16:27:16 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger

Posted by "Austin Nobis (Code Review)" <ge...@cloudera.org>.
Austin Nobis has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/12914 )

Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger
......................................................................

IMPALA-8226: Add grant/revoke to/from group for Ranger

This patch adds fupport for GRANT privilege statements to GROUP and
REVOKE privilege statements from GROUP.  The grammar has been updated to
support FROM GROUP and TO GROUP for GRANT/REVOKE statements, i.e:

GRANT <privilege> ON <resource> TO GROUP <group>
REVOKE <privilege> ON <resource> FROM GROUP <group>

Currently, only Ranger's authorization implementation supports GROUP
based privileges. Sentry will throw an UnsupportedOperationException if
it is the enabled authorization provider and this new grammar is used.

Testing:
- AuthorizationStmtTest was updated to also test for GROUP
  authorization.
- ToSqlTest was updated to test for GROUP changes to the grammar.
- A GROUP based E2E test was added to test_ranger.py
- Ran all FE tests
- Ran authorization E2E tests

Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
---
M common/thrift/CatalogObjects.thrift
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/org/apache/impala/authorization/AuthorizationManager.java
M fe/src/main/java/org/apache/impala/authorization/NoneAuthorizationFactory.java
M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java
M fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java
M fe/src/main/java/org/apache/impala/authorization/sentry/SentryImpaladAuthorizationManager.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/test/java/org/apache/impala/analysis/AuthorizationStmtTest.java
M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java
M fe/src/test/resources/ranger-hive-security.xml
M testdata/bin/create-load-data.sh
A testdata/cluster/ranger/setup/impala_group.json.template
M testdata/cluster/ranger/setup/impala_user.json.template
M tests/authorization/test_ranger.py
15 files changed, 212 insertions(+), 83 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/12914/3
-- 
To view, visit http://gerrit.cloudera.org:8080/12914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
Gerrit-Change-Number: 12914
Gerrit-PatchSet: 3
Gerrit-Owner: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>

[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12914 )

Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger
......................................................................


Patch Set 7: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/12914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
Gerrit-Change-Number: 12914
Gerrit-PatchSet: 7
Gerrit-Owner: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Wed, 03 Apr 2019 21:17:26 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12914 )

Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger
......................................................................


Patch Set 5:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/2625/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/12914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
Gerrit-Change-Number: 12914
Gerrit-PatchSet: 5
Gerrit-Owner: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Wed, 03 Apr 2019 20:54:19 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12914 )

Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger
......................................................................


Patch Set 2:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/2618/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/12914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
Gerrit-Change-Number: 12914
Gerrit-PatchSet: 2
Gerrit-Owner: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Tue, 02 Apr 2019 23:52:28 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12914 )

Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger
......................................................................


Patch Set 7:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3977/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/12914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
Gerrit-Change-Number: 12914
Gerrit-PatchSet: 7
Gerrit-Owner: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Wed, 03 Apr 2019 21:17:27 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger

Posted by "Fredy Wijaya (Code Review)" <ge...@cloudera.org>.
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/12914 )

Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger
......................................................................


Patch Set 6: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/12914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
Gerrit-Change-Number: 12914
Gerrit-PatchSet: 6
Gerrit-Owner: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Wed, 03 Apr 2019 21:17:10 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8226: Add grant/revoke to/from group for Ranger

Posted by "Austin Nobis (Code Review)" <ge...@cloudera.org>.
Austin Nobis has posted comments on this change. ( http://gerrit.cloudera.org:8080/12914 )

Change subject: IMPALA-8226: Add grant/revoke to/from group for Ranger
......................................................................


Patch Set 6:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/12914/5/common/thrift/CatalogObjects.thrift
File common/thrift/CatalogObjects.thrift:

http://gerrit.cloudera.org:8080/#/c/12914/5/common/thrift/CatalogObjects.thrift@485
PS5, Line 485: // Represents a type of principal.
             : enum TPrincipalType {
> nit: remove this comment i don't think this is specific to Sentry anymore.
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/12914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I28b7b3e4c776ad1bb5bdc184c7d733d0b5ef5e96
Gerrit-Change-Number: 12914
Gerrit-PatchSet: 6
Gerrit-Owner: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Austin Nobis <an...@cloudera.com>
Gerrit-Reviewer: Fredy Wijaya <fw...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: radford nguyen <ra...@gmail.com>
Gerrit-Comment-Date: Wed, 03 Apr 2019 21:14:48 +0000
Gerrit-HasComments: Yes