[jira] [Commented] (RANGER-3685) hive 'show' sql produces excessive audit log

["Madhan Neethiraj (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/RANGER-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17615409#comment-17615409 ] 

Madhan Neethiraj commented on RANGER-3685:
------------------------------------------

[~Xuze Yang]  - RANGER-3000 introduced audit-filtering feature, which enables to skip generation of audit logs for conditions specified in service-configuration. Default service-configuration for Hive was updated in RANGER-3214 (shown below) to skip auditing of METADATA OPERATIONS, like SHOW DATABASES. Note the following text in Hive service-def configuration: 
{code:java}
{'actions':['METADATA OPERATION'], 'isAudited': false}{code}
 

 

 
{code:java}
 {
    "itemId":        6,
    "name":         "ranger.plugin.audit.filters",
    "type":         "string",
    "mandatory":    false,
    "label":        "Ranger Default Audit Filters",
    "defaultValue": "[ {'accessResult': 'DENIED', 'isAudited': true}, {'actions':['METADATA OPERATION'], 'isAudited': false}, {'users':['hive','hue'],'actions':['SHOW_ROLES'],'isAudited':false} ]"
  }{code}
 

Can you please verify the configuration of your Hive service (like dev_hive) if appropriate value is specified for "ranger.plugin.audit.filters".

> hive 'show' sql produces excessive audit log
> --------------------------------------------
>
>                 Key: RANGER-3685
>                 URL: https://issues.apache.org/jira/browse/RANGER-3685
>             Project: Ranger
>          Issue Type: Improvement
>          Components: audit
>    Affects Versions: 2.1.0
>            Reporter: Xuze Yang
>            Assignee: Xuze Yang
>            Priority: Major
>         Attachments: RANGER-3685-hive-show-sql-produces-excessive-audit-l.patch
>
>
> Since ranger2.1.0. For "show databases", user needs any permission on Database to get authorized. RangerHiveAuthorizer.filterListCmdObjects() is implemented to filter out the database which user don't have access to. 
> This is a good implementation, but a problem comes with it：the method will record an audit log for each database(each table for "show tables"). In our production environment, There are 80,000 tables under a database of hive. A show tables operation will generate 80001(The extra one is the verification of USE permissions) audit logs. Unfortunately, our customers will frequently call the show tables operation.
> This brings up two problems: 
>  # Valuable audit logs are flooded
>  # Take up a lot of storage resources
> For problem.2, such a scenario has occurred in our environment: our audit log destination is down. All audit logs are spooled in disk files, several days later, the size of the disk file exceeded 800G, causing other components to fail to provide services normally.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)