[jira] [Commented] (NIFI-190) HoldFile processor

["Bryan Bende (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14701278#comment-14701278 ] 

Bryan Bende commented on NIFI-190:
----------------------------------

If I moved forward on a controller service for this, does anyone disagree with an interface like the following?

{code}
public interface WaitNotifyService {

  boolean containsKey(String key);

  void put(String key, Map<String,String> attributes)

  Map<String,String> get(String key);

  void remove(String key)

  // removes any signals older than the given timestamp
  // alternatively could provide a method to obtain an iterator
  void expire(long timestamp) 

}
{code}

This interface assumes that the overall design of the HoldFile processor remains the same, and it just gets split into Wait and Notify (better names chime in). Meaning that Wait keeps re-queuing a FlowFile until waitNotifyService.containsKey(...) == true, and Notify uses waitNotifyService.put(...) to send a signal.

If we want more semantic meaning on the interface, I could see calling "containsKey" something like "releasable", and "put" could be called "notify".

> HoldFile processor
> ------------------
>
>                 Key: NIFI-190
>                 URL: https://issues.apache.org/jira/browse/NIFI-190
>             Project: Apache NiFi
>          Issue Type: New Feature
>          Components: Extensions
>            Reporter: Joseph Gresock
>            Assignee: Bryan Bende
>            Priority: Minor
>         Attachments: HoldFile_example.xml
>
>
> Our team has developed a processor for the following use case:
> * Format A needs to be sent to Endpoint A
> * Format B needs to be sent to Endpoint B, but should not proceed until A has reached Endpoint A.  We most commonly have this restriction when Endpoint B requires some output of Endpoint A.
> The proposed HoldFile processor takes 2 types of flow files as input:
> * Files to be held
> * Signal files that can release corresponding held files, based on the value of a configurable "release" attribute
> Signal files are distinguished from held files by the presence of the "flow.file.release.value" attribute.  The processor is configured with a "Release Signal Attribute".  Held files with this attribute whose value matches a received signal value will be released.
> An example:
> HoldFile is configured with Release Signal Attribute = "myId".  Its 'Hold' relationship routes back onto itself.
> 1. flowFile 1 { myId : "123" } enters HoldFile.  It is routed to the 'Hold' relationship
> 2. flowFile 2 { flow.file.release.value : "123" } enters HoldFile.  flowfile 1 is then routed to 'Release', and flow file 2 is removed from the session.
> Signal flow files will also copy their attributes to matching held files, unless otherwise indicated.  This is what allows the output of Endpoint A to pass to Endpoint B, above.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)