[jira] [Updated] (METRON-668) Remove the "tickUpdate" profile config and make the "init" phase not reset variables

["Casey Stella (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/METRON-668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Casey Stella updated METRON-668:
--------------------------------
    Fix Version/s: 0.3.1

> Remove the "tickUpdate" profile config and make the "init" phase not reset variables
> ------------------------------------------------------------------------------------
>
>                 Key: METRON-668
>                 URL: https://issues.apache.org/jira/browse/METRON-668
>             Project: Metron
>          Issue Type: Improvement
>            Reporter: Casey Stella
>            Assignee: Casey Stella
>             Fix For: 0.3.1
>
>
> Originally during work on the MAD outlier work, I conceived of a need for a new callback in the profile configuration, "tickUpdate" that ran at the tick and had the variables from the tick that just completed available to it.  This was done so that I could merge the state accumulated in the tick with a lookback window of state for MAD.  The problem is that the "init" phase happens after this and blows away the changes done in "tickUpdate", so it never worked like intended.
> It occurs to me that what we really want is not to have two separate config phases, but only one, "init" and to not reset the variables on the tick for the profile.  You can, of course, choose to update them by overwriting them in the "init" phase *or* you can choose to use them as part of your init.
> For context, this would make the example for MAD:
> {code:javascript}
> {
>   "profiles": [
>     {
>       "profile": "sketchy_mad",
>       "foreach": "'global'",
>       "onlyif": "true",
>       "init" : {
>         "s": "OUTLIER_MAD_STATE_MERGE(PROFILE_GET('sketchy_mad',
> 'global', 5, 'MINUTES'))"
>                },
>       "tickUpdate": {
>         "s": "OUTLIER_MAD_STATE_MERGE(PROFILE_GET('sketchy_mad',
> 'global', 5, 'MINUTES'), s)"
>                 },
>       "update": {
>         "s": "OUTLIER_MAD_ADD(s, value)"
>                 },
>       "result": "s"
>     }
>   ]
> }
> {code}
> is functionally equivalent to
> {code:javascript}
> {
>   "profiles": [
>     {
>       "profile": "sketchy_mad",
>       "foreach": "'global'",
>       "onlyif": "true",
>       "init" : {
>         "s": "OUTLIER_MAD_STATE_MERGE(PROFILE_GET('sketchy_mad',
> 'global', 5, 'MINUTES'))"
>                },
>       "update": {
>         "s": "OUTLIER_MAD_ADD(s, value)"
>                 },
>       "result": "s"
>     }
>   ]
> }
> {code}
> This resets the MAD state to the last 5 minute window.  If we did NOT reset the state and keep accumulating state (provided we did not clear the variables on init, we could do the following:
> {code:javascript}
> {
>   "profiles": [
>     {
>       "profile": "sketchy_mad",
>       "foreach": "'global'",
>       "onlyif": "true",
>       "init" : {
>         "s": "if exists(s) then s else OUTLIER_MAD_STATE_MERGE(PROFILE_GET('sketchy_mad',
> 'global', 5, 'MINUTES'))"
>                },
>       "update": {
>         "s": "OUTLIER_MAD_ADD(s, value)"
>                 },
>       "result": "s"
>     }
>   ]
> }
> {code}
> s would get initialized sensibly and then always accumulate as long as the topology continued (rather than having a fixed lookback).
> In short, making init to not reset the variables shouldn't cause any harm and should provide another set of use-cases for the profiler.  Also, tickUpdate has no function whatsoever and should be removed because it gets overwritten by init directly after being called.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)