[jira] [Commented] (NIFI-6762) New Processor - Watch Multiple S3 Buckets for content changes

["Joey Frazee (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-6762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17028204#comment-17028204 ] 

Joey Frazee commented on NIFI-6762:
-----------------------------------

[~aramCodez] Question: is there something we get here that the S3 event notification + read from queue pattern or just creating two processors and creating a funnel or two connections to the downstream flow doesn't provide?

> New Processor - Watch Multiple S3 Buckets for content changes
> -------------------------------------------------------------
>
>                 Key: NIFI-6762
>                 URL: https://issues.apache.org/jira/browse/NIFI-6762
>             Project: Apache NiFi
>          Issue Type: New Feature
>          Components: Extensions
>    Affects Versions: 1.9.2
>            Reporter: Aram S Openden
>            Priority: Major
>              Labels: Processor, S3, watch
>   Original Estimate: 384h
>  Remaining Estimate: 384h
>
> Proposing New S3 processor that is a "variation" on the existing ListS3 Processor. For lack of better term, I will call this "Watch Multiple S3 Buckets Processor" (naming here is open to feedback). This new Processor takes an input attribute "Bucket" from the FlowFile, which could use NiFi ExpressionLanguage  to compute that value. Hence, the Processor could "watch" different buckets in an S3 Object Store.
> This processor must handle the State Management logic for multiple S3 buckets. The simplest solution for this problem add to modify the State Management logic to prepend the bucket name to this key/objectName that is being used to track state.
> Finally, the Dev should document the maximum number of buckets it will monitor at once, maximum amount of state/storage it can have to track the various listings in various buckets.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)