[jira] [Assigned] (METRON-534) Metron Should have service to monitor HDFS for changes to files

["Otto Fowler (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/METRON-534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Otto Fowler reassigned METRON-534:
----------------------------------

    Assignee: Otto Fowler

> Metron Should have service to monitor HDFS for changes to files
> ---------------------------------------------------------------
>
>                 Key: METRON-534
>                 URL: https://issues.apache.org/jira/browse/METRON-534
>             Project: Metron
>          Issue Type: New Feature
>            Reporter: Otto Fowler
>            Assignee: Otto Fowler
>            Priority: Minor
>         Attachments: inotify-ascii.png
>
>
> With the INotify library it is possible to receive notifications from HDFS on file changes. 
>  A service that utilized this and was configured with "rules" to process notifications on files of interest ( configurations, rules etc ) in the system would be able to send notifications to Zookeeper
>  This may allow for things that are better stored in HDFS to remain there, and not force the system to put things in Zookeeper just for change notifications.
> Issues:
>  * Inotify requires hdfs super user, what are the implications on a secure cluster etc?
>  * performance / scalability and load
>  * Can it be deployed with yarn?
>  * Is Zookeeper actually the right place for all the configurations
> ---------------------------------------------------------------------
>  
> !inotify-ascii.png!
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)