[jira] [Commented] (IMPALA-9625) Impala's COMPUTE STATS statement generates duplicate ALTER events

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-9625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17165090#comment-17165090 ] 

ASF subversion and git services commented on IMPALA-9625:
---------------------------------------------------------

Commit d83074c5992d51d136a509639a038bde5c3393bf in impala's branch refs/heads/master from Fang-Yu Rao
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=d83074c ]

IMPALA-9977: Remove duplicate Ranger audit log entries for ALTER events

This JIRA could be considered as a follow-up to IMPALA-9625, where we
converted the name of a TAccessEvent to lowercase to avoid duplicate
audits in the Set used to maintain the collected TAccessEvent's so that
there will not be duplicate TAccessEvent's in the file specified by the
flag of "-audit_event_log_dir" when Impala is started.

However, the patch for IMPALA-9625 only considered the audits that are
exported to the specific file mentioned above but not the
PrivilegeRequest's that will be processed by Ranger which in turn would
produce the corresponding audit log entries. Therefore, the
fully-qualified table name that is provided when
Analyzer#registerPrivReq() is called in Analyzer#getTable() is not
necessarily in lowercase, resulting in duplicate AuthzAuditEvent's
stored in the corresponding RangerBufferAuditHandler because the
full table names returned from registerAuthAndAuditEvent() and
getTable() differ. Refer to IMPALA-9625 for more details.

To resolve the inconsistencies, this patch converts the arguments of
database and table names to lowercase when
PrivilegeRequestBuilder#onTable() is building the corresponding
PrivilegeRequest, which will later be added to the Set of
PrivilegeRequest's for Ranger to process.

Testing:
- Added an FE test in RangerAuditLogTest.java to make sure no duplicate
  Ranger audit log entries are produced.
- Verified that the patch passes the exhaustive tests in the DEBUG
  build.

Change-Id: Iab9b664ad5ee9722182007ee67d14bf47bd03d8a
Reviewed-on: http://gerrit.cloudera.org:8080/16231
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>


> Impala's COMPUTE STATS statement generates duplicate ALTER events
> -----------------------------------------------------------------
>
>                 Key: IMPALA-9625
>                 URL: https://issues.apache.org/jira/browse/IMPALA-9625
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Frontend
>            Reporter: Fang-Yu Rao
>            Assignee: Fang-Yu Rao
>            Priority: Critical
>
> Impala's COMPUTE STATS statement results in the registration of the ALTER event twice. One is in {{Analyzer#registerAuthAndAuditEvent()}} at [https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/analysis/Analyzer.java#L3131-L3133] and the other is in {{Analyzer#getTable()}} at [https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/analysis/Analyzer.java#L2862-L2863].
> In {{registerAuthAndAuditEvent()}}, the corresponding full table name {{table.getFullName()}} is produced by a call to {{Analyzer#resolveTableRef()}} ([https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java#L352]). The resulting database and table names are both in lowercase.
> However, in {{getTable()}}, the fully-qualified table name is produce by a call to {{Analyzer#getFqTableName()}} at [https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/analysis/Analyzer.java#L2836]. The resulting database and table names are in their originally unconverted form provided by the user from the Impala shell. Hence, there is no guarantee that the database and table names are both in lowercase.
> Therefore, if a user does not provide lowercase database and table names, the returned full table name from {{registerAuthAndAuditEvent()}} and {{getTable()}} would differ, resulting in duplicate ALTER events for the same table.
> We should at least make the full table name consistent every time when we register such an audit event to avoid duplicate entries in the log.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org