[jira] [Commented] (EAGLE-14) Re-assemble high level hdfs commands for better policy

["Edward Zhang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/EAGLE-14?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15044374#comment-15044374 ] 

Edward Zhang commented on EAGLE-14:
-----------------------------------

Status: created a new class called HdfsUserCommandReassembler which accepts event from Spout and output event to FileSensitivity join unit. The idea is to figure out user command by analyzing its constituting hdfs audit logs and using Siddhi pattern match. But because getfileinfo audit log is very basic log and shared by most of high level user command, it is not correct to use getfileinfo as part of pattern. It will cause ambiguity otherwise.

Today I make this re-assembler to be configurable by fie or by eagle database and now only supports 3 high level commands, read, appendToFile and copyFromLocal. We can continue to test to support more high level command.

> Re-assemble high level hdfs commands for better policy
> ------------------------------------------------------
>
>                 Key: EAGLE-14
>                 URL: https://issues.apache.org/jira/browse/EAGLE-14
>             Project: Eagle
>          Issue Type: Improvement
>    Affects Versions: 0.3.0
>            Reporter: Edward Zhang
>            Assignee: Edward Zhang
>             Fix For: 0.3.0
>
>   Original Estimate: 240h
>  Remaining Estimate: 240h
>
> User today can use Eagle to create policy against the finest granularity of hdfs command for example getFileInfo, list, delete etc., but that sometimes is not good for defining a policy against a user's native command.
> Hdfs client would interpret user's high level command into multiple fine commands and send those fine commands to Hdfs server which in turn are logged into namenode audit log. For example one copyToLocal high-level command probably involves multiple fine commands like rename, move, delete etc.
> Eagle should have a capability to re-assemble those command in real-time to let user define policy on those high-level commands.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)