You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Quanlong Huang (Jira)" <ji...@apache.org> on 2021/03/17 06:27:00 UTC
[jira] [Comment Edited] (IMPALA-10401) Enable Ranger Audit logs in
minicluster
[ https://issues.apache.org/jira/browse/IMPALA-10401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17303138#comment-17303138 ]
Quanlong Huang edited comment on IMPALA-10401 at 3/17/21, 6:26 AM:
-------------------------------------------------------------------
By adding the following configs in fe/src/test/resources/ranger-hive-audit.xml, we can enabled audit logs to log4j:
{code:xml}
<property>
<name>xasecure.audit.is.enabled</name>
<value>true</value>
</property>
<property>
<name>xasecure.audit.destination.log4j</name>
<value>true</value>
</property>
{code}
However, it's just audits in the client side. E.g. we can find these in impalad.INFO after executing a query:
{code:java}
I0317 14:19:16.222668 18674 Log4JAuditDestination.java:107] {"repoType":3,"repo":"test_impala","reqUser":"quanlong","evtTime":"2021-03-17 14:19:16.089","access":"select","resource":"functional/alltypestiny","resType":"@table","action":"select","result":1,"agent":"impala","policy":2,"enforcer":"ranger-acl","cliIP":"::ffff:127.0.0.1","reqData":"select id from functional.alltypestiny","agentHost":"quanlong-OptiPlex-BJ","logType":"RangerAudit","id":"662c97a5-0cb5-42e0-bd09-13e8473ef0f7-0","seq_num":0,"event_count":1,"event_dur_ms":0,"tags":[],"cluster_name":"test-cluster","policy_version":2207}
I0317 14:19:16.222872 18674 Log4JAuditDestination.java:107] {"repoType":3,"repo":"test_impala","reqUser":"quanlong","evtTime":"2021-03-17 14:19:16.089","access":"select","resource":"functional/alltypestiny/id","resType":"@column","action":"select","result":1,"agent":"impala","policy":2,"enforcer":"ranger-acl","cliIP":"::ffff:127.0.0.1","reqData":"select id from functional.alltypestiny","agentHost":"quanlong-OptiPlex-BJ","logType":"RangerAudit","id":"f788e4f9-8689-4fb9-a4d0-afd4f5e50a2e-0","seq_num":1,"event_count":1,"event_dur_ms":0,"tags":[],"cluster_name":"test-cluster","policy_version":2207}
{code}
The "policy" field shows which policy takes effect. The "result" field shows whether the access is allowed.
For debugging e2e test like IMPALA-10587, we still need the access audits about policies. Something that's shown in Ranger Admin Web UI: Audits->Admin
!Ranger Admin Access Audit.png|width=780,height=367!
I think we should either dumping the results of it or enable DEBUG logging for ranger.
Refs:
[https://cwiki.apache.org/confluence/display/RANGER/Ranger+0.5+Audit+Configuration#Ranger0.5AuditConfiguration-AudittoLog4j]
[https://tomcat.apache.org/tomcat-8.0-doc/config/valve.html]
was (Author: stiga-huang):
By adding the following configs in fe/src/test/resources/ranger-hive-audit.xml, we can enabled audit logs to log4j:
{code:xml}
<property>
<name>xasecure.audit.is.enabled</name>
<value>true</value>
</property>
<property>
<name>xasecure.audit.destination.log4j</name>
<value>true</value>
</property>
{code}
However, it's just audits in the client side. E.g. we can find these in impalad.INFO after executing a query:
{code:java}
I0317 14:19:16.222668 18674 Log4JAuditDestination.java:107] {"repoType":3,"repo":"test_impala","reqUser":"quanlong","evtTime":"2021-03-17 14:19:16.089","access":"select","resource":"functional/alltypestiny","resType":"@table","action":"select","result":1,"agent":"impala","policy":2,"enforcer":"ranger-acl","cliIP":"::ffff:127.0.0.1","reqData":"select id from functional.alltypestiny","agentHost":"quanlong-OptiPlex-BJ","logType":"RangerAudit","id":"662c97a5-0cb5-42e0-bd09-13e8473ef0f7-0","seq_num":0,"event_count":1,"event_dur_ms":0,"tags":[],"cluster_name":"test-cluster","policy_version":2207}
I0317 14:19:16.222872 18674 Log4JAuditDestination.java:107] {"repoType":3,"repo":"test_impala","reqUser":"quanlong","evtTime":"2021-03-17 14:19:16.089","access":"select","resource":"functional/alltypestiny/id","resType":"@column","action":"select","result":1,"agent":"impala","policy":2,"enforcer":"ranger-acl","cliIP":"::ffff:127.0.0.1","reqData":"select id from functional.alltypestiny","agentHost":"quanlong-OptiPlex-BJ","logType":"RangerAudit","id":"f788e4f9-8689-4fb9-a4d0-afd4f5e50a2e-0","seq_num":1,"event_count":1,"event_dur_ms":0,"tags":[],"cluster_name":"test-cluster","policy_version":2207}
{code}
The "policy" field shows which policy takes effect. The "result" field shows whether the access is allowed.
For debugging e2e test like IMPALA-10587, we still need the access audits about policies. Something that's shown in Ranger Admin Web UI: Audits->Admin
!Ranger Admin Access Audit.png|width=780,height=367!
I think we should either dumping the results of it or enable DEBUG logging for ranger.
> Enable Ranger Audit logs in minicluster
> ---------------------------------------
>
> Key: IMPALA-10401
> URL: https://issues.apache.org/jira/browse/IMPALA-10401
> Project: IMPALA
> Issue Type: New Feature
> Components: Infrastructure
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Major
> Attachments: Ranger Admin Access Audit.png
>
>
> It'd be helpful for debugging e2e ranger authz tests if we can get the audit logs. Ranger supports ingesting audit logs to Solr or HDFS files. We currently don't have Solr in our minicluster. We can try enabling ranger audit logs in HDFS and save the file for each job.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org