You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Fang-Yu Rao (Code Review)" <ge...@cloudera.org> on 2022/08/15 16:39:18 UTC

[Impala-ASF-CR] IMPALA-11494: Don't always produce Ranger audit log for authorized query

Fang-Yu Rao has uploaded this change for review. ( http://gerrit.cloudera.org:8080/18850


Change subject: IMPALA-11494: Don't always produce Ranger audit log for authorized query
......................................................................

IMPALA-11494: Don't always produce Ranger audit log for authorized query

Before this patch, when Impala could not resolve a given table
'<db_name>.<tbl_name>' during the query analysis, Impala would still
attempt to register 2 privilege requests. One was for the table
'<tbl_name>' under the database '<db_name>' and the other was for the
table '<db_name>' under the database 'default'. The first one should be
registered since Impala had to determine whether such an access should
be allowed (even though in fact the table did not exist), whereas the
second one was incorrect in that 'default.<db_name>' definitely was not
'<db_name>.<tbl_name>' in general.

Furthermore, Impala always sent audit log entries to the Ranger server
for an authorized query against non-existing table(s).

The 2 facts described above resulted in Impala producing Ranger audit
log entries for the tables '<db_name>.<tbl_name>' and
'default.<db_name>' when a requesting user granted the corresponding
privileges on the databases of '<db_name>' and 'default' submitted a
query against a non-existing table '<db_name>.<tbl_name>'. None of the
audit log entries should be generated because i) the privilege request
for '<db_name>.<tbl_name>' was allowed and '<db_name>.<tbl_name>' did
not exist, i.e., '<db_name>.<tbl_name>' was not accessed at all, and ii)
'default.<db_name>' did not correspond to a table.

This patch fixes the 2 issues mentioned above so that Impala will not
generate any Ranger audit log entry for an authorized query against a
non-existing table.

Testing:
 - Added a frontend test case to verify no Ranger audit log entry will
   be produced for an authorized query against a non-existing table.

Change-Id: I701652e457d3118f43249e83be933713b17ce48f
---
M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java
M fe/src/main/java/org/apache/impala/authorization/BaseAuthorizationChecker.java
M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java
M fe/src/test/java/org/apache/impala/authorization/AuthorizationTestBase.java
M fe/src/test/java/org/apache/impala/authorization/ranger/RangerAuditLogTest.java
7 files changed, 67 insertions(+), 20 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/50/18850/1
-- 
To view, visit http://gerrit.cloudera.org:8080/18850
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I701652e457d3118f43249e83be933713b17ce48f
Gerrit-Change-Number: 18850
Gerrit-PatchSet: 1
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>

[Impala-ASF-CR] IMPALA-11494: Don't always produce Ranger audit log for authorized query

Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/18850 )

Change subject: IMPALA-11494: Don't always produce Ranger audit log for authorized query
......................................................................


Patch Set 4: Code-Review+2

Thanks for the changes!


-- 
To view, visit http://gerrit.cloudera.org:8080/18850
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I701652e457d3118f43249e83be933713b17ce48f
Gerrit-Change-Number: 18850
Gerrit-PatchSet: 4
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Wed, 17 Aug 2022 16:45:53 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11494: Don't always produce Ranger audit log for authorized query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18850 )

Change subject: IMPALA-11494: Don't always produce Ranger audit log for authorized query
......................................................................


Patch Set 4: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/18850
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I701652e457d3118f43249e83be933713b17ce48f
Gerrit-Change-Number: 18850
Gerrit-PatchSet: 4
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Wed, 17 Aug 2022 21:51:52 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11494: Don't always produce Ranger audit log for authorized query

Posted by "Aman Sinha (Code Review)" <ge...@cloudera.org>.
Aman Sinha has posted comments on this change. ( http://gerrit.cloudera.org:8080/18850 )

Change subject: IMPALA-11494: Don't always produce Ranger audit log for authorized query
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/18850/1/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java
File fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java:

http://gerrit.cloudera.org:8080/#/c/18850/1/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@200
PS1, Line 200:     if (authzOk && !analysisOk) auditHandler.getAuthzEvents().clear();
If this condition is met, is it necessary to call flush() below ?



-- 
To view, visit http://gerrit.cloudera.org:8080/18850
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I701652e457d3118f43249e83be933713b17ce48f
Gerrit-Change-Number: 18850
Gerrit-PatchSet: 1
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Tue, 16 Aug 2022 00:16:07 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11494: Don't always produce Ranger audit log for authorized query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18850 )

Change subject: IMPALA-11494: Don't always produce Ranger audit log for authorized query
......................................................................


Patch Set 3:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/11165/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/18850
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I701652e457d3118f43249e83be933713b17ce48f
Gerrit-Change-Number: 18850
Gerrit-PatchSet: 3
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Tue, 16 Aug 2022 20:07:17 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11494: Don't always produce Ranger audit log for authorized query

Posted by "Fang-Yu Rao (Code Review)" <ge...@cloudera.org>.
Fang-Yu Rao has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/18850 )

Change subject: IMPALA-11494: Don't always produce Ranger audit log for authorized query
......................................................................

IMPALA-11494: Don't always produce Ranger audit log for authorized query

Before this patch, when Impala could not resolve a given table
'<db_name>.<tbl_name>' during the query analysis, Impala would still
attempt to register 2 privilege requests. One was for the table
'<tbl_name>' under the database '<db_name>' and the other was for the
table '<db_name>' under the database 'default'. The first one should be
registered since Impala had to determine whether such an access should
be allowed (even though in fact the table did not exist), whereas the
second one was incorrect in that 'default.<db_name>' definitely was not
'<db_name>.<tbl_name>' in general.

Furthermore, Impala always sent audit log entries to the Ranger server
for an authorized query against non-existing table(s).

The 2 facts described above resulted in Impala producing Ranger audit
log entries for the tables '<db_name>.<tbl_name>' and
'default.<db_name>' when a requesting user granted sufficient privileges
on the databases of '<db_name>' and 'default' submitted a query against
a non-existing table '<db_name>.<tbl_name>'. None of the audit log
entries should be generated because i) the privilege request for
'<db_name>.<tbl_name>' was allowed and '<db_name>.<tbl_name>' did not
exist, and ii) 'default.<db_name>' did not correspond to a table.

This patch fixes the 2 issues mentioned above so that Impala will not
generate any Ranger audit log entry for an authorized query against a
non-existing table.

Testing:
 - Added a frontend test case to verify no Ranger audit log entry will
   be produced for an authorized query against a non-existing table.

Change-Id: I701652e457d3118f43249e83be933713b17ce48f
---
M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java
M fe/src/main/java/org/apache/impala/authorization/BaseAuthorizationChecker.java
M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java
M fe/src/test/java/org/apache/impala/authorization/AuthorizationTestBase.java
M fe/src/test/java/org/apache/impala/authorization/ranger/RangerAuditLogTest.java
7 files changed, 75 insertions(+), 21 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/50/18850/4
-- 
To view, visit http://gerrit.cloudera.org:8080/18850
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I701652e457d3118f43249e83be933713b17ce48f
Gerrit-Change-Number: 18850
Gerrit-PatchSet: 4
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>

[Impala-ASF-CR] IMPALA-11494: Don't always produce Ranger audit log for authorized query

Posted by "Fang-Yu Rao (Code Review)" <ge...@cloudera.org>.
Fang-Yu Rao has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/18850 )

Change subject: IMPALA-11494: Don't always produce Ranger audit log for authorized query
......................................................................

IMPALA-11494: Don't always produce Ranger audit log for authorized query

Before this patch, when Impala could not resolve a given table
'<db_name>.<tbl_name>' during the query analysis, Impala would still
attempt to register 2 privilege requests. One was for the table
'<tbl_name>' under the database '<db_name>' and the other was for the
table '<db_name>' under the database 'default'. The first one should be
registered since Impala had to determine whether such an access should
be allowed (even though in fact the table did not exist), whereas the
second one was incorrect in that 'default.<db_name>' definitely was not
'<db_name>.<tbl_name>' in general.

Furthermore, Impala always sent audit log entries to the Ranger server
for an authorized query against non-existing table(s).

The 2 facts described above resulted in Impala producing Ranger audit
log entries for the tables '<db_name>.<tbl_name>' and
'default.<db_name>' when a requesting user granted sufficient privileges
on the databases of '<db_name>' and 'default' submitted a query against
a non-existing table '<db_name>.<tbl_name>'. None of the audit log
entries should be generated because i) the privilege request for
'<db_name>.<tbl_name>' was allowed and '<db_name>.<tbl_name>' did not
exist, and ii) 'default.<db_name>' did not correspond to a table.

This patch fixes the 2 issues mentioned above so that Impala will not
generate any Ranger audit log entry for an authorized query against a
non-existing table.

Testing:
 - Added a frontend test case to verify no Ranger audit log entry will
   be produced for an authorized query against a non-existing table.

Change-Id: I701652e457d3118f43249e83be933713b17ce48f
---
M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java
M fe/src/main/java/org/apache/impala/authorization/BaseAuthorizationChecker.java
M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java
M fe/src/test/java/org/apache/impala/authorization/AuthorizationTestBase.java
M fe/src/test/java/org/apache/impala/authorization/ranger/RangerAuditLogTest.java
7 files changed, 71 insertions(+), 21 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/50/18850/2
-- 
To view, visit http://gerrit.cloudera.org:8080/18850
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I701652e457d3118f43249e83be933713b17ce48f
Gerrit-Change-Number: 18850
Gerrit-PatchSet: 2
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>

[Impala-ASF-CR] IMPALA-11494: Don't always produce Ranger audit log for authorized query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18850 )

Change subject: IMPALA-11494: Don't always produce Ranger audit log for authorized query
......................................................................


Patch Set 4:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/11172/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/18850
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I701652e457d3118f43249e83be933713b17ce48f
Gerrit-Change-Number: 18850
Gerrit-PatchSet: 4
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Wed, 17 Aug 2022 16:54:22 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11494: Don't always produce Ranger audit log for authorized query

Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/18850 )

Change subject: IMPALA-11494: Don't always produce Ranger audit log for authorized query
......................................................................


Patch Set 3:

(1 comment)

lgtm, one comment about code cleanness

http://gerrit.cloudera.org:8080/#/c/18850/3/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java
File fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java:

http://gerrit.cloudera.org:8080/#/c/18850/3/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@203
PS3, Line 203:     if (authzOk && !analysisOk) auditHandler.getAuthzEvents().clear();
             : 
             :     // We do not send audit log entries to the Ranger server when authorization succeeded
             :     // but analysis failed.
             :     if (!authzOk || analysisOk) auditHandler.flush();
The two conditions are the opposites of each other - can you convert this to if/else blocks? I think that it would be cleaner.



-- 
To view, visit http://gerrit.cloudera.org:8080/18850
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I701652e457d3118f43249e83be933713b17ce48f
Gerrit-Change-Number: 18850
Gerrit-PatchSet: 3
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Wed, 17 Aug 2022 09:06:46 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11494: Don't always produce Ranger audit log for authorized query

Posted by "Fang-Yu Rao (Code Review)" <ge...@cloudera.org>.
Fang-Yu Rao has posted comments on this change. ( http://gerrit.cloudera.org:8080/18850 )

Change subject: IMPALA-11494: Don't always produce Ranger audit log for authorized query
......................................................................


Patch Set 3:

> Uploaded patch set 3.

I slightly updated the code comment in patch set 3.


-- 
To view, visit http://gerrit.cloudera.org:8080/18850
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I701652e457d3118f43249e83be933713b17ce48f
Gerrit-Change-Number: 18850
Gerrit-PatchSet: 3
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Tue, 16 Aug 2022 19:46:18 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11494: Don't always produce Ranger audit log for authorized query

Posted by "Fang-Yu Rao (Code Review)" <ge...@cloudera.org>.
Fang-Yu Rao has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/18850 )

Change subject: IMPALA-11494: Don't always produce Ranger audit log for authorized query
......................................................................

IMPALA-11494: Don't always produce Ranger audit log for authorized query

Before this patch, when Impala could not resolve a given table
'<db_name>.<tbl_name>' during the query analysis, Impala would still
attempt to register 2 privilege requests. One was for the table
'<tbl_name>' under the database '<db_name>' and the other was for the
table '<db_name>' under the database 'default'. The first one should be
registered since Impala had to determine whether such an access should
be allowed (even though in fact the table did not exist), whereas the
second one was incorrect in that 'default.<db_name>' definitely was not
'<db_name>.<tbl_name>' in general.

Furthermore, Impala always sent audit log entries to the Ranger server
for an authorized query against non-existing table(s).

The 2 facts described above resulted in Impala producing Ranger audit
log entries for the tables '<db_name>.<tbl_name>' and
'default.<db_name>' when a requesting user granted sufficient privileges
on the databases of '<db_name>' and 'default' submitted a query against
a non-existing table '<db_name>.<tbl_name>'. None of the audit log
entries should be generated because i) the privilege request for
'<db_name>.<tbl_name>' was allowed and '<db_name>.<tbl_name>' did not
exist, and ii) 'default.<db_name>' did not correspond to a table.

This patch fixes the 2 issues mentioned above so that Impala will not
generate any Ranger audit log entry for an authorized query against a
non-existing table.

Testing:
 - Added a frontend test case to verify no Ranger audit log entry will
   be produced for an authorized query against a non-existing table.

Change-Id: I701652e457d3118f43249e83be933713b17ce48f
---
M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java
M fe/src/main/java/org/apache/impala/authorization/BaseAuthorizationChecker.java
M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java
M fe/src/test/java/org/apache/impala/authorization/AuthorizationTestBase.java
M fe/src/test/java/org/apache/impala/authorization/ranger/RangerAuditLogTest.java
7 files changed, 73 insertions(+), 21 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/50/18850/3
-- 
To view, visit http://gerrit.cloudera.org:8080/18850
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I701652e457d3118f43249e83be933713b17ce48f
Gerrit-Change-Number: 18850
Gerrit-PatchSet: 3
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>

[Impala-ASF-CR] IMPALA-11494: Don't always produce Ranger audit log for authorized query

Posted by "Fang-Yu Rao (Code Review)" <ge...@cloudera.org>.
Fang-Yu Rao has posted comments on this change. ( http://gerrit.cloudera.org:8080/18850 )

Change subject: IMPALA-11494: Don't always produce Ranger audit log for authorized query
......................................................................


Patch Set 1:

Hi all, please let me know if you have any comment on the patch. Thank you very much for the help!


-- 
To view, visit http://gerrit.cloudera.org:8080/18850
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I701652e457d3118f43249e83be933713b17ce48f
Gerrit-Change-Number: 18850
Gerrit-PatchSet: 1
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Mon, 15 Aug 2022 16:39:46 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11494: Don't always produce Ranger audit log for authorized query

Posted by "Fang-Yu Rao (Code Review)" <ge...@cloudera.org>.
Fang-Yu Rao has posted comments on this change. ( http://gerrit.cloudera.org:8080/18850 )

Change subject: IMPALA-11494: Don't always produce Ranger audit log for authorized query
......................................................................


Patch Set 4:

(1 comment)

> Patch Set 3:
> 
> (1 comment)
> 
> lgtm, one comment about code cleanness

Hi all, I have slightly revised the patch set 3 according to Csaba's suggestion. Please let me know if there is any other comment

Thank you very much for the help!

http://gerrit.cloudera.org:8080/#/c/18850/3/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java
File fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java:

http://gerrit.cloudera.org:8080/#/c/18850/3/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@203
PS3, Line 203:       // to determine whether the correct audit events are collected.
             :       auditHandler.getAuthzEvents().clear();
             :     } else {
             :       // We send audit log entries to the Ranger server only if authorization failed or
             :       // analysis succeeded.
> The two conditions are the opposites of each other - can you convert this t
Thanks Csaba!

I will change the above to an if-else block in the next patch.



-- 
To view, visit http://gerrit.cloudera.org:8080/18850
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I701652e457d3118f43249e83be933713b17ce48f
Gerrit-Change-Number: 18850
Gerrit-PatchSet: 4
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Wed, 17 Aug 2022 16:35:15 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11494: Don't always produce Ranger audit log for authorized query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/18850 )

Change subject: IMPALA-11494: Don't always produce Ranger audit log for authorized query
......................................................................

IMPALA-11494: Don't always produce Ranger audit log for authorized query

Before this patch, when Impala could not resolve a given table
'<db_name>.<tbl_name>' during the query analysis, Impala would still
attempt to register 2 privilege requests. One was for the table
'<tbl_name>' under the database '<db_name>' and the other was for the
table '<db_name>' under the database 'default'. The first one should be
registered since Impala had to determine whether such an access should
be allowed (even though in fact the table did not exist), whereas the
second one was incorrect in that 'default.<db_name>' definitely was not
'<db_name>.<tbl_name>' in general.

Furthermore, Impala always sent audit log entries to the Ranger server
for an authorized query against non-existing table(s).

The 2 facts described above resulted in Impala producing Ranger audit
log entries for the tables '<db_name>.<tbl_name>' and
'default.<db_name>' when a requesting user granted sufficient privileges
on the databases of '<db_name>' and 'default' submitted a query against
a non-existing table '<db_name>.<tbl_name>'. None of the audit log
entries should be generated because i) the privilege request for
'<db_name>.<tbl_name>' was allowed and '<db_name>.<tbl_name>' did not
exist, and ii) 'default.<db_name>' did not correspond to a table.

This patch fixes the 2 issues mentioned above so that Impala will not
generate any Ranger audit log entry for an authorized query against a
non-existing table.

Testing:
 - Added a frontend test case to verify no Ranger audit log entry will
   be produced for an authorized query against a non-existing table.

Change-Id: I701652e457d3118f43249e83be933713b17ce48f
Reviewed-on: http://gerrit.cloudera.org:8080/18850
Reviewed-by: Csaba Ringhofer <cs...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java
M fe/src/main/java/org/apache/impala/authorization/BaseAuthorizationChecker.java
M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java
M fe/src/test/java/org/apache/impala/authorization/AuthorizationTestBase.java
M fe/src/test/java/org/apache/impala/authorization/ranger/RangerAuditLogTest.java
7 files changed, 75 insertions(+), 21 deletions(-)

Approvals:
  Csaba Ringhofer: Looks good to me, approved
  Impala Public Jenkins: Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/18850
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I701652e457d3118f43249e83be933713b17ce48f
Gerrit-Change-Number: 18850
Gerrit-PatchSet: 5
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>

[Impala-ASF-CR] IMPALA-11494: Don't always produce Ranger audit log for authorized query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18850 )

Change subject: IMPALA-11494: Don't always produce Ranger audit log for authorized query
......................................................................


Patch Set 4:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8457/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/18850
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I701652e457d3118f43249e83be933713b17ce48f
Gerrit-Change-Number: 18850
Gerrit-PatchSet: 4
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Wed, 17 Aug 2022 17:03:50 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11494: Don't always produce Ranger audit log for authorized query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18850 )

Change subject: IMPALA-11494: Don't always produce Ranger audit log for authorized query
......................................................................


Patch Set 2:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/11164/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/18850
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I701652e457d3118f43249e83be933713b17ce48f
Gerrit-Change-Number: 18850
Gerrit-PatchSet: 2
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Tue, 16 Aug 2022 20:02:16 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11494: Don't always produce Ranger audit log for authorized query

Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/18850 )

Change subject: IMPALA-11494: Don't always produce Ranger audit log for authorized query
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/18850/1/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java
File fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java:

http://gerrit.cloudera.org:8080/#/c/18850/1/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@198
PS1, Line 198:     // When the query was authorized, we do not send any audit log entry to the Ranger
             :     // server when there was an AnalysisException during query analysis.
I am a bit unsure about this case - when should we produce audit logs, when a the user actually uses a resource (e.g. table), or when he/she tries to access them? For example what will happen in case of an explain statement? I don't necessarily see that as different than a failed analyses.

Do you know how Hive works in this case?



-- 
To view, visit http://gerrit.cloudera.org:8080/18850
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I701652e457d3118f43249e83be933713b17ce48f
Gerrit-Change-Number: 18850
Gerrit-PatchSet: 1
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Tue, 16 Aug 2022 05:51:56 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11494: Don't always produce Ranger audit log for authorized query

Posted by "Aman Sinha (Code Review)" <ge...@cloudera.org>.
Aman Sinha has posted comments on this change. ( http://gerrit.cloudera.org:8080/18850 )

Change subject: IMPALA-11494: Don't always produce Ranger audit log for authorized query
......................................................................


Patch Set 3: Code-Review+1

+1 from my side.  Will wait for Csaba's response before bumping to +2


-- 
To view, visit http://gerrit.cloudera.org:8080/18850
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I701652e457d3118f43249e83be933713b17ce48f
Gerrit-Change-Number: 18850
Gerrit-PatchSet: 3
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Tue, 16 Aug 2022 21:56:18 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11494: Don't always produce Ranger audit log for authorized query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18850 )

Change subject: IMPALA-11494: Don't always produce Ranger audit log for authorized query
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/11155/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/18850
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I701652e457d3118f43249e83be933713b17ce48f
Gerrit-Change-Number: 18850
Gerrit-PatchSet: 1
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Mon, 15 Aug 2022 17:00:21 +0000
Gerrit-HasComments: No