[jira] [Commented] (NIFI-1037) Hdfs Inotify processor

["Joseph Witt (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-1037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14987176#comment-14987176 ] 

Joseph Witt commented on NIFI-1037:
-----------------------------------

wow - i missed this ticket in the email stream.  But this is awesome...  We should definitely support this.  We have ListHDFS and FetchHDFS.  This inotify driven listing could be very valuable.  Great idea

> Hdfs Inotify processor
> ----------------------
>
>                 Key: NIFI-1037
>                 URL: https://issues.apache.org/jira/browse/NIFI-1037
>             Project: Apache NiFi
>          Issue Type: New Feature
>            Reporter: nicolas maillard
>            Priority: Minor
>             Fix For: 0.4.0
>
>
> HDFS has an Inotify interface that enables to access the HDFS edit stream.
> https://issues.apache.org/jira/browse/HDFS-6634
> Creating a processor to listen in and get notifications either for select directories or select actions would have many applications.
> - Stream to a search engine the activity on HDFS
> - Wait for specific actions or files to trigger workflows, like duplication to other clusters
> - Validate ingestion processes
> etc..
> probably more I don't think of.
> I have a first working beta version that needs to evolve
> it reuses the Hadoop-nar-bundle
> Needs a HDFS 2.7  dependency currently done through editing the Hadop-lib bundle
> let me know if this idea makes sense and would be of interest to the community
> would love to contribute the idea



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)