[jira] [Commented] (NIFI-1582) New processor to update attributes with state

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-1582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15726306#comment-15726306 ] 

ASF GitHub Bot commented on NIFI-1582:
--------------------------------------

Github user mattyb149 commented on a diff in the pull request:

    https://github.com/apache/nifi/pull/319#discussion_r91141036
  
    --- Diff: nifi-nar-bundles/nifi-update-attribute-bundle/nifi-update-attribute-processor/src/main/java/org/apache/nifi/processors/attributes/UpdateAttribute.java ---
    @@ -162,20 +146,33 @@ public ValidationResult validate(String subject, String input, ValidationContext
         // static properties
         public static final PropertyDescriptor DELETE_ATTRIBUTES = new PropertyDescriptor.Builder()
                 .name("Delete Attributes Expression")
    -            .description("Regular expression for attributes to be deleted from flowfiles.")
    +            .description("Regular expression for attributes to be deleted from FlowFiles.")
                 .required(false)
                 .addValidator(DELETE_PROPERTY_VALIDATOR)
                 .expressionLanguageSupported(true)
                 .build();
     
    -    // relationships
    -    public static final Relationship REL_SUCCESS = new Relationship.Builder()
    -            .description("All FlowFiles are routed to this relationship").name("success").build();
    +    public static final PropertyDescriptor STORE_STATE = new PropertyDescriptor.Builder()
    +            .name("Store State")
    +            .description("Select whether or not state will be stored. Selecting 'Stateless' will offer the default functionality of purely updating the attributes on a " +
    +                    "FlowFile in a stateless manner. Selecting 'Stateful' will not only store the attributes on the FlowFile but also in the Processors state. See the 'Stateful Usage' " +
    +                    "topic of the 'Additional Details' section of this processor's documentation for more information")
    +            .required(true)
    +            .allowableValues(DO_NOT_STORE_STATE, STORE_STATE_LOCALLY)
    +            .defaultValue(DO_NOT_STORE_STATE)
    +            .build();
    +    public static final PropertyDescriptor STATEFUL_VARIABLES_INIT_VALUE = new PropertyDescriptor.Builder()
    +            .name("Stateful Variables Initial Value")
    +            .description("If using state to set/reference variables then this value is used to set the initial value of the stateful variable. This will only be used in the @OnScheduled method " +
    +                    "when state does not contain a value for the variable.")
    +            .required(false)
    +            .defaultValue("0")
    --- End diff --
    
    Should this be empty rather than zero? Seems like many of the use cases are numeric in nature so I see the value of a zero default, but wondering for the most general case if it should default to empty. I'm fine either way, just wanted your thoughts on it.


> New processor to update attributes with state
> ---------------------------------------------
>
>                 Key: NIFI-1582
>                 URL: https://issues.apache.org/jira/browse/NIFI-1582
>             Project: Apache NiFi
>          Issue Type: New Feature
>            Reporter: Joseph Percivall
>            Assignee: Joseph Percivall
>
> This idea was sparked by a thread on the user list and should allow basic data science:
> I expect that in the future I’ll need something a little more sophisticated but for now my problem is very simple:
> I want to be able to trigger an alert (only once) when an attribute in an incoming stream, for instance, goes over a predefined threshold. The Processor should then trigger (only once again) another trigger when the signal goes back to normal (below threshold). Basically a RouteByAttribute but with memory.
> Thanks 
> Claudio
> ------------------------------------------------
> Hello Claudio,
> Your use-case actually could leverage a couple of recently added features to create a really cool open-source processor. The two key features that were added are State Management and the ability to reference processor specific variables in expression language. You can take a look at RouteText to see both in action. 
> By utilizing both you can create a processor that is configured with multiple Expression language expressions. There would be dynamic properties which would accept expression language and then store the evaluated value via state management. Then there would be a routing property (that supports expression language) that could simply add an attribute to the flowfile with the evaluated value which would allow it to be used by flowing processors for routing.
> This would allow you to do your use-case where you store the value for the incoming stream and route differently once you go over a threshold. It could even allow more complex use-cases. One instance, I believe, would be possible is to have a running average and standard deviation and route data to different locations based on it's standard deviation.
> You can think of this like an UpdateAttribute with the ability to store and calculate variables using expression language.
> Joe



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)